Company Description
Experian is a global data and technology company that empowers people and businesses around the world. We operate in many markets, such as financial services, healthcare, automotive, agribusiness, insurance, and others. Experian invests in people and in new advanced technologies to unlock the power of data. We have an amazing team of 25,200 employees in 32 countries.
Our uniqueness is valuing yours. Experian's culture, which is people-centered, inclusive, and purpose-driven, is recognized by several awards, including World's Best Workplaces™ 2025 (Fortune Global Top 25) and Great Place To Work™ in 26 countries, among others. Check out Experian Life on social media or explore our careers site to understand why. Experian also takes pride in being an employer that promotes equal opportunities and affirmative action.
Job Description
We are looking for a Senior Data Engineer to work with large volumes of data in a highly scalable environment, mainly using Apache Spark/Python on Databricks. This professional will be responsible for developing, optimizing, and maintaining data ingestion pipelines, with a focus on modern and efficient architectures.
Main Activities Include
- Building and maintaining ingestion pipelines using Autoloader (incremental, schema evolution, throughput optimization).
- Large-scale data processing using Spark Structured Streaming and Spark SQL.
- Implementing data engineering best practices, such as partitioning and optimization with Delta Lake.
- Knowledge of git version control and pull requests.
- Analysis, improvement, and optimization of costs and performance in Databricks.
- Implementing Data Quality.
- Creating clear documentation of requirements.
- Collaborating with business and analytics teams to ensure data is structured and available securely.
Qualifications
- Experience with Databricks for processing, transforming, and analyzing large volumes of data.
- Solid knowledge of ETL/ELT processes, ensuring the integration, quality, and availability of information.
- Experience in Data Engineering, including developing, maintaining, and optimizing scalable pipelines.
- Hands-on experience in data architecture and modeling, ensuring efficient structuring of information.
- Experience integrating data from multiple sources and systems.
- Experience working in Cloud environments and Big Data ecosystems.
- Knowledge of best practices in data governance, quality, security, and performance.
- Ability to develop robust and scalable solutions aligned with business needs.
- Analytical profile, with the ability to identify opportunities for improvement and optimization of data processes.
- Ability to work easily in multidisciplinary teams and translate business requirements into efficient technical solutions.
Additional Information
Serasa Experian is much more than you imagine. With the purpose of creating a better future by expanding opportunities for people and businesses, in Brazil we are more than 4,000 people working across various teams and specialties. Here, every skill and diversity complements each other, and you can work on what you love most. We are committed to building an inclusive culture and an environment where people can balance their careers with their commitments and personal interests, valuing well-being.
We are deeply committed to being one of the best and most innovative companies to work for in the country, enabling amazing experiences and careers for our people. Our strong people-first approach is recognized externally through several market certifications: we were awarded by Great Place To Work™ in 24 countries and by the international Top Employers certification, and we are also recognized as one of the best companies for young professionals and have a 4.6 rating on Glassdoor. Each recognition tells us we are on the right path, providing an increasingly better work environment for our talent.
Experian Careers - Creating a better tomorrow together
Find out what its like to work for Experian by clicking here