Data Engineer

Posted on February 27, 2025

Apply Now

Job Description

  • About the Role:
  • Duration: 6 months
  • Location: Remote
  • Timings: Full Time (As per company timings)
  • Notice Period: (Immediate Joiner - Only)
  • Experience: 6-7 Years
  • About the Role:
  • We are looking for a Data Engineer with expertise in ETL pipelines, data integration, and
  • cloud infrastructure. This role focuses on building and maintaining data pipelines,
  • ensuring data integrity, and optimizing data processing workflows. You will work
  • closely with AI and backend teams to support data-driven decision-making and scalable
  • infrastructure.
  • Responsibilities:
  • Data Engineering & Pipeline Development:
  • Develop and maintain ETL pipelines to integrate data from various sources, including Snowflake, Salesforce, and external APIs (e.g., Google/Facebook Ads, Zillow, Rently).
  • Ensure data quality, consistency, and integrity across multiple systems.
  • Optimize data ingestion and transformation workflows to improve efficiency and scalability.
  • Implement batch and real-time data processing solutions using Python, SQL, and Spark.
  • Work on lead ingestion and listing syndication with platforms such as Zillow, Facebook Ads, and Rently.
  • Cloud Infrastructure & DevOps:
  • Deploy and manage cloud-based data processing solutions in AWS/Azure environments.
  • Optimize CI/CD pipelines for efficient deployment and data pipeline automation.
  • Monitor data pipeline performance and implement alerts for failures (AWS CloudWatch, Datadog, Slack notifications).
  • Maintain data security, governance, and compliance with best practices.
  • Collaboration & Optimization:
  • Work closely with AI and backend teams to provide clean, structured data for machine learning models.
  • Review and optimize database queries and data storage solutions for performance.
  • Document data engineering processes and ensure smooth handovers for internal teams.
  • Key Technologies & Skills:
  • Must-Have:
  • ETL & Data Pipelines: Snowflake, SQL, Python, Apache Airflow, PySpark.
  • Cloud Platforms: AWS (Lambda, S3, DynamoDB, Glue) / Azure (Data Factory, Functions).
  • Database Management: PostgreSQL, Snowflake, Redshift.
  • Big Data Processing: Spark, Kafka, Hadoop.
  • DevOps & Automation: CI/CD pipelines, Terraform, Docker, Kubernetes.
  • Monitoring & Logging: AWS CloudWatch, Datadog, ELK stack.
  • Good-to-Have:
  • Streaming Data Processing: Apache Flink, Kinesis.
  • Data Security & Compliance: IAM, GDPR, HIPAA best practices.
  • Data Warehousing & Optimization: Query tuning, indexing, caching strategies.

Required Skills

etl & data pipelines: snowflake sql python apache airflow pyspark. cloud platforms: aws (lambda s3 dynamodb glue) / azure (data factory functions). database management: postgresql snowflake redshift. big data processing: spark kafka hadoop. devops & automation: ci/cd pipelines terraform docker kubernetes. monitoring & logging: aws cloudwatch datadog elk stack.