Course Outline
NiFi and Data Flow Fundamentals
- Data in motion versus data at rest: underlying concepts and challenges
- NiFi architecture: core components, flow controller, provenance, and bulletin board
- Essential elements: processors, connections, controllers, and provenance tracking
Big Data Context and Integration
- NiFi’s role within Big Data ecosystems (Hadoop, Kafka, cloud storage)
- Introduction to HDFS, MapReduce, and contemporary alternatives
- Application cases: stream ingestion, log shipping, and event pipelines
Installation, Configuration, and Cluster Setup
- Installing NiFi in both single-node and cluster modes
- Configuring clusters: node roles, ZooKeeper integration, and load balancing
- Orchestrating NiFi deployments using Ansible, Docker, or Helm
Designing and Managing Dataflows
- Implementing routing, filtering, splitting, and merging operations
- Processor configuration (e.g., InvokeHTTP, QueryRecord, PutDatabaseRecord)
- Managing schema, data enrichment, and transformation tasks
- Error management, retry mechanisms, and backpressure handling
Integration Scenarios
- Connecting to databases, messaging systems, and REST APIs
- Streaming data to analytics platforms like Kafka, Elasticsearch, or cloud storage
- Integrating with Splunk, Prometheus, or other logging pipelines
Monitoring, Recovery, and Provenance
- Utilizing the NiFi UI, metrics, and the provenance visualizer
- Creating autonomous recovery strategies and graceful failure protocols
- Managing backups, flow versioning, and change control
Performance Tuning and Optimization
- Adjusting JVM, heap size, thread pools, and clustering parameters
- Refining flow design to minimize bottlenecks
- Applying resource isolation, flow prioritization, and throughput control
Best Practices and Governance
- Flow documentation, naming conventions, and modular design principles
- Security measures: TLS, authentication, access control, and data encryption
- Change control, versioning, role-based access, and audit trails
Troubleshooting and Incident Response
- Addressing common issues such as deadlocks, memory leaks, and processor errors
- Analyzing logs, diagnosing errors, and investigating root causes
- Recovery techniques and flow rollback procedures
Hands-on Lab: Realistic Data Pipeline Implementation
- Constructing an end-to-end flow covering ingestion, transformation, and delivery
- Implementing error handling, backpressure, and scaling mechanisms
- Conducting performance testing and pipeline tuning
Summary and Next Steps
Requirements
- Familiarity with the Linux command line
- Foundational knowledge of networking and data systems
- Basic exposure to data streaming or ETL concepts
Target Audience
- System administrators
- Data engineers
- Developers
- DevOps professionals
Testimonials (7)
Hands on exercises. Class should have been 5 days, but the 3 days helped to clear up a lot of questions that I had from working with NiFi already
James - BHG Financial
Course - Apache NiFi for Administrators
I thought the trainer's pace was good. He left no student behind with his approach. He was very supportive with us NEWBIES that may not have had a System Administrator or Infrastructure role during our career or resurrected those skills from a prior period of our career.
Pamdrea Ivory - BHG Financial
Course - Apache NiFi for Administrators
I like the hands on section. It helped me to better retain information by completing the provided exercises. Also, the trainer's ability to engage with the entire class made me feel comfortable to ask questions on things I was not sure about.
Leila - BHG Financial
Course - Apache NiFi for Administrators
Use-cases, examples for building NiFi dataflows. We worked on troubleshooting common problems and gotchas.
Nelson - BHG Financial
Course - Apache NiFi for Administrators
I loved the structure. We dove into the basics of Nifi, concepts, use cases, etc. on day 1. On day 2, we got to put Day 1 knowledge into practice by building out flows to meet scenario requirements. Day 3 we got to see Nifi Registry and version control, mulit-tenancy, and go over Q&A.
Adam - BHG Financial
Course - Apache NiFi for Administrators
Very little, I found it very difficult, especially because I fell behind and didn't attend the first sessions.
Rolando Garcia - OIT para Mexico y Cuba
Course - Apache NiFi for Administrators
Machine Translated
Knowing that there is a program with many potential applications for the work I do
Jorge Alberto Martinez - OIT para Mexico y Cuba
Course - Apache NiFi for Administrators
Machine Translated
