Company Description
Experian is a global data and technology company that empowers people and businesses around the world. We operate in diverse markets, such as financial services, health, automotive, agribusiness, insurance, among others. Experian invests in people and in new advanced technologies to unleash the power of data. We have an incredible team of 25,200 employees in 32 countries.
Our uniqueness is valuing yours. Experian's culture, centered on people, inclusive and purpose-driven, is recognized by various awards, including World's Best Workplaces™ 2025 (Fortune Global Top 25) and Great Place To Work™ in 26 countries, among others. Check out Experian Life on social media or explore our careers site to understand why. Experian is also proud to be an employer that promotes equal opportunities and affirmative action. If you have any disability or need that requires adaptation, please let us know as soon as possible.
Job Description
We are looking for a Senior Data Engineer to work on the construction and evolution of our modern data platform, focused on large-scale data ingestion, processing and availability. The professional will be responsible for developing robust and scalable pipelines using Databricks, contributing to the reliability, governance and efficiency of the company's data ecosystem.
We are seeking a hands-on profile, with strong technical capability, architectural vision and experience in highly complex corporate data environments.
Main responsibilities:
- Develop and maintain batch and streaming data ingestion pipelines using Databricks.
- Design and implement data integration solutions from multiple sources (APIs, relational databases, files, legacy systems and SaaS).
- Build and optimize processes using Apache Spark.
- Apply data engineering best practices, DataOps and observability.
- Ensure quality, reliability and performance of pipelines.
- Work on data modeling and organization in Lakehouse architectures.
- Work in partnership with business, analytics, governance and platform teams.
- Support the definition of technical standards, reusable frameworks and development best practices.
- Lead troubleshooting and performance optimizations in critical data processes.
- Take part in modernization and evolution initiatives for the data platform.
Qualifications
- Solid experience as a Data Engineer.
- Proven experience with Databricks in production environments.
- Experience with Apache Spark (PySpark or Scala).
- Experience developing data ingestion and transformation pipelines.
- Hands-on experience with Data Lake and Lakehouse solutions.
- Experience with Git and CI/CD processes.
- Knowledge of analytical data modeling.
Technical knowledge:
- Advanced Python.
- Advanced SQL.
- Databricks Workflows.
- Delta Lake.
- Spark Structured Streaming.
- REST APIs.
- Modern data architectures.
- Spark performance optimization.
- Pipeline control and observability.
- Data governance and quality concepts.
Nice to have:
- Databricks certifications.
- Experience with Unity Catalog.
- Hands-on experience with Terraform and Infrastructure as Code.
- Knowledge of Kafka or Event Hubs.
- Experience with monitoring using Datadog.
- Knowledge of Airflow.
Behavioral competencies:
- Strong sense of ownership.
- Ability to work autonomously and proactively.
- Collaborative profile focused on problem solving.
- Good communication with technical and business areas.
- Ability to influence technical decisions.
- Continuous improvement mindset.
- Systemic vision and focus on scalability.
Additional Information
Our uniqueness is that we celebrate yours. Experian's people first, inclusive and purpose driven culture is multi award-winning; World's Best Workplaces™ 2025 (Fortune Global Top 25), Great Place To Work™ in 26 countries to name a few. Check out Experian Life on social or explore our Careers Site to understand why. Experian is also proud to be an Equal Opportunity and Affirmative Action employer. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.
Experian Careers - Creating a better tomorrow together
Find out what its like to work for Experian by clicking here