Senior Data Engineer – Bir Ecosystem
İşin təsviri
- Degree in computer science, information science, engineering, mathematics, or related technical discipline
- 5+ years experience with SQL and NoSQL technologies
- Strong experience working with Oracle and PostgreSQL databases
- Extensive experience in ETL development, with strong skills in Python programming
- Advanced knowledge of PySpark for large-scale distributed data processing
- Proficient in Airflow for scheduling, orchestrating and monitoring complex ETL/ELT workflows
- Expertise in Kafka for event streaming and messaging pipelines
- Experience working with MinIO (S3-compatible storage), managing data in Parquet and Iceberg formats
- Familiarity with Lakehouse concepts
- Deep understanding of MPP architecture concepts
The following experience will be considered as an advantage:
- Familiarity with Impala for interactive SQL querying on Big Data
- Experience with Greenplum
- Familiarity with containerization (Docker, Kubernetes)
- Experience with CDC tools like Debezium and Oracle GoldenGate for real-time data replication
- Experience with data visualization tools (e.g. Tableau, PowerBI, etc.)
- Understanding of DataOps and DevOps principles
- Knowledge of data modeling and database design
- Experience with data engineering best practices, such as data security, data access control and data governance
- Knowledge of Grafana for monitoring metrics
- Proficient with GitLab for version control
- Experience mentoring junior data engineers and collaborating with cross-functional teams (Data Scientists, Analysts, DevOps)
Obligations
- Designing a data lake to process and store an array of (un)structured data loaded from raw sources. Development of the data model – properly store the data and access it as needed for business purposes
- Development of the integration process – development of integration with various systems so that they can have a single view of key indicators when making decisions
- Data preparation and ETL – development of a pipeline for extracting, transforming and loading data Developing and monitoring data pipelines to ensure data quality and integrity
- Automation and optimization of the data transformation process
- Working closely with other teams in the organization to ensure data is available and accessible for analytics and decision-making
Benefits
Opportunities for professional growth and development
Competitive salary and bonuses
Comprehensive insurance coverage
Supportive work environment
Visa Premium salary card
Corporate discounts and events
Additional vacation days
Discounted education and employee loans
New cozy studio office in Port Baku Tower 2
Strong teamwork with Agile principles, regular team buildings and themed events
Multicultural environment with foreign colleagues sharing their best experiences
