Поиск работы на robota.uaukraine

Эта вакансия уже завершена

Sigma Software — вакансия в Senior Big Data Engineer (AdTech)
Вакансия закрыта

Senior Big Data Engineer (AdTech)

Sigma Software
2 года назад
29 мая 2021
Днепрул. вулиця Січеславська Набережна, 53.

Are you thrilled about designing the right architecture for a new client and kicking it off at full scale in production? Do you enjoy putting a strong team together and lead it towards a project’s success? Are you keen on sharing your knowledge and educating people around you? Then you are very likely to enjoy this role!

Being our team member means being open minded, proactive, and friendly, supporting, ready to accept the challenge, take responsibility, and never stop learning.

Customer

We are working with a rapidly growing US AdTech company. Founded by three ex-Googlers, it has a highly technical team and an excellent technical culture.

Project

The project is about building the next generation of real-time bidding software that enables sophisticated marketers to break free from the limitations and constraints of opaque, one-size-fits-all programmatic buying platforms. Most of the ETL is written in Python and managed by Airflow DAGs.

The Kinesis application is written in Java and handles a variety of scaling and configuration challenges. Having an excellent working knowledge of SQL is critical as we do a number of the ETL steps in Snowflake, and a poorly written query could have a significant performance and cost impact.

You will cooperate closely with Big Data team on a customer’s side to improve a platform looking into new features and become more efficient performance-wise. The team currently works on several tools and applications. By joining our team, you would work with some or all of them:

  • Manage and build on a high scale event parsing and recording system. We use Kinesis to handle billions of events and ship them to S3, Snowflake, databases, and a variety of other logs, both internal and external
  • Manage and build on a set of ETL pipelines that move terabytes of data through Snowflake
  • Operate multiple services that provide real-time data flows to our internal systems (both the UI and the optimization engines)

Responsibilities:

  • Contributing to new technologies investigations and complex solutions design
  • Coming up with well-designed technical solutions and robust code
  • Working and professionally communicating with the customer’s team
  • Taking up responsibility for delivering major solution features
  • Participating in requirements gathering and clarification process
  • Developing core modules and functions
  • Performing code reviews, writing unit, and integration tests

Requirements:

  • Solid experience with Java or Python
  • Solid knowledge in algorithms and data structures
  • Experience with AWS technologies — Glue, Lambda, Step Functions, S3, etc.
  • Experience with developing data pipelines based on one of the mainstream frameworks like Spark/Flink/Presto, etc.
  • Experience with developing data lakes and data warehouses based on one of the mainstream technologies like Hive/Snowflake/OLAP/Hudi, etc.
  • Knowledge in SQL and solid experience with NoSQL or RDBMS technologies
  • Experience with data integrations and different data formats CSV, Protobuf, Parquet, Avro, ORC, etc.
  • Solid understanding of technical excellence and hands-on experience with code reviews, test development, and CI/CD

Would be a plus:

  • Experience with developing Snowflake-driven data warehouses
  • Experience with developing Kinesis-driven streaming data pipelines

Sigma Software

Sigma Software — вакансия в Senior Big Data Engineer (AdTech): фото 2

We are IT consulting and software product company.

больше 500  сотрудников

с 2002 года  на рынке

  • Скидки сотрудникамСкидки сотрудникам
  • Корпоративные мероприятияКорпоративные мероприятия
  • Компенсация спортзалаКомпенсация спортзала
  • Компенсация обученияКомпенсация обучения
  • Гибкий графикГибкий график

Похожие вакансии по городам:

Похожие вакансии по профессиям: