Data Engineer - Python & Microservices

We are seeking an experienced Data Engineer with strong expertise in Python, Microservices Architecture, Data Transformation, and Backend Engineering to join our Data Engineering team. The ideal candidate will be responsible for designing and developing scalable backend services, data processing frameworks, RESTful APIs, and data transformation pipelines that support enterprise data-driven applications.

This role requires strong hands-on experience with Python-based web frameworks, database technologies, ETL pipelines, microservices, API security, and modern software engineering practices. Candidates should have experience building production-grade applications with a focus on scalability, reliability, security, and performance.

Requirements

Key Responsibilities

• Design, develop, and maintain scalable backend applications and web frameworks using Python.
• Build robust RESTful APIs and microservices to serve enterprise datasets and business applications.
• Develop and optimize data transformation pipelines to process, clean, enrich, and prepare data for analytics and business consumption.
• Design and implement scalable data access layers integrating multiple relational and NoSQL databases.
• Collaborate with frontend developers to integrate user-facing components with backend services.
• Ensure applications are secure, highly available, performant, and maintainable.
• Design and maintain database schemas and data models aligned with business requirements.
• Implement ETL processes for extracting, transforming, and loading large-scale datasets.
• Optimize database performance, query execution, and data retrieval mechanisms.
• Troubleshoot production issues and implement scalable solutions for long-term platform stability.
• Participate in code reviews, testing activities, technical design discussions, and engineering best practices.
• Work closely with business stakeholders, architects, and engineering teams within Agile delivery environments.

Required Technical Skills

Python Development

• Strong hands-on experience in Core Python development.
• Expertise in building backend services and APIs using Python.
• Strong understanding of object-oriented programming principles and software design patterns.
• Experience writing clean, reusable, maintainable, and production-quality code.

Web Frameworks & API Development

Strong experience with one or more Python frameworks including:

• FastAPI
• Flask
• Django
• Pyramid

Experience with:

• RESTful API design and development
• API versioning and documentation
• Routing and middleware implementation
• Authentication and authorization
• Framework testing and debugging

Microservices Architecture

• Strong experience designing and developing microservices-based applications.
• Deep understanding of distributed systems architecture.
• Experience building scalable and resilient service-oriented platforms.
• Knowledge of service communication patterns and API orchestration.
• Experience handling concurrent requests and high-volume transaction processing.

Asynchronous Programming

• Strong understanding of asyncio and asynchronous programming concepts.
• Experience handling concurrent connections and non-blocking operations.
• Knowledge of performance optimization for I/O-intensive applications.

Security & API Protection

• Strong understanding of API and microservices security best practices.
• Experience implementing:

• OAuth 2.0
• JWT Authentication
• API Gateway integrations
• Encryption and secure communication protocols
  • Experience implementing authentication, authorization, and access control mechanisms.

Database & ORM Expertise

• Strong experience with relational and NoSQL databases.
• Experience designing database schemas and optimizing database performance.
• Strong proficiency with:

• SQL
• PostgreSQL
• MySQL
• MongoDB
• Redis

Hands-on experience with ORMs including:

• SQLAlchemy
• Django ORM

Ability to:

• Define models
• Perform CRUD operations
• Write optimized queries
• Handle transactions
• Manage database migrations

Data Engineering & Transformation

• Experience building and maintaining ETL pipelines.
• Strong understanding of data transformation, enrichment, aggregation, and cleansing techniques.
• Experience ensuring data quality, consistency, and integrity.
• Ability to support analytical and data-driven applications through scalable processing frameworks.

DevOps & CI/CD

• Experience building and maintaining CI/CD pipelines.
• Experience deploying and supporting production-grade applications.
• Understanding of infrastructure automation and modern software delivery practices.
• Experience with release management and deployment workflows.

Big Data & Data Technologies

Exposure to technologies such as:

• Hadoop
• Apache Spark
• Apache Kafka

Understanding of:

• Data Warehousing concepts
• Distributed data processing
• Enterprise data architectures

Version Control & Development Practices

• Strong experience with Git version control.
• Experience with:

• Branching strategies
• Code reviews
• Pull Requests
• Merge conflict resolution
• Collaborative development workflows

Testing & Debugging

• Experience with:

• Unit Testing
• Integration Testing
• Logging
• Monitoring
• Performance Troubleshooting
• Production Support

Nice to Have

• Experience with AWS, Azure, or GCP.
• Experience with Docker and Kubernetes.
• Exposure to cloud-native architectures.
• Knowledge of data governance, compliance, privacy, and security standards.
• Experience supporting large-scale enterprise data platforms.

Required Competencies

• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management capabilities.
• Strong collaboration and teamwork skills.
• Ability to work effectively in Agile/Scrum environments.
• Strong ownership mindset with attention to detail.
• Ability to document solutions and communicate technical concepts clearly.

Education

Bachelor’s Degree in Computer Science, Information Technology, Engineering, Data Engineering, or a related field.