At Talkdesk, our Engineering team follows a micro-service architecture approach to build the next generation of Talkdesk, with vertical teams responsible for all the decisions under their services. Through our Agile Coaches, we promote agile and collaborative practices, we are huge fans of Scrum, pair programming and we won’t let a single line of code reach production without peer code reviews. We strongly believe that the only true authority stems from knowledge, not from position and we always treat others with respect, deference and patience.
We are looking for a savvy Senior Data Engineer to join our growing team of Data experts. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The Data Engineering will partner with Business Analysts, Data Scientists, and fellow software engineers to build data pipelines and solve data-related problems within the team.
Develop, deploy and maintain Big Data solutions that will ingest, process and store the necessary data to power Talkdesk’s Data Science, BI and Analytics solutions.
Design batch or streaming dataflows capable of processing large quantities of fast moving unstructured data;
Monitoring dataflows and underlying systems, promoting the necessary changes to ensure scalable, high performance solutions and assure data quality and availability.
Work closely with data scientists and the rest of Talkdesk’s engineering to deliver world class data driven solutions.
Proficient in one or more languages like Java, Python, Kotlin, Elixir, Go, Ruby
Strong understanding of distributed computing principles and with distributed systems;
Building stream-processing systems, using solutions such as Spark-Streaming, Flink, Kafka Streams, Storm or similar;
Experience with Big Data processing frameworks such as Hadoop, Spark or Samza;
Good knowledge of Big Data analytical tools, such as Hive, Impala, Presto or Drill;
Experience with integration of data from multiple data sources;
Experience with traditional RDBMS and data modeling.
Experience with Data Warehouses and related concepts. Knowledge of Redshift or other Data Warehousing solutions.
Experience with NoSQL databases, such as MongoDB, Cassandra, and HBase;
Knowledge of ETL techniques and frameworks, such as Flume;
Experience with messaging systems, such as Kafka or RabbitMQ;
Experience with cloud environments such as AWS or Google Cloud;
Strong written and verbal English communication skills.
Nice to have / Pluses:
BS/MS Degree in Computer Engineering, Computer Science, Applied Math, or an other engineering-related field;
Experience in Agile development methodology/Scrum;
Experience in securing data access through frameworks like Apache Sentry, AWS IAM, Kerberos or others;
Good understanding of Lambda and Kappa Architectures, along with their advantages and drawbacks;
Experience with Cloudera/MapR/Hortonworks;
Management of Hadoop, Spark, Flink clusters with all included services;