Data Engineer - Cloud Platform

Location Singapore
Discipline Information & Communications Technology
Job Reference BBBH144248_1728036123
Salary Negotiable
Consultant Name Sravanthi Gurram
Consultant Email [email protected]
Consultant Contact No. 6232 8818
EA License No. 02C3423
Consultant Registration No. R2197596


Responsibilities:

  • Design and build data pipelines: Create the infrastructure to collect, process, and store data from various sources.
  • Develop data assets platform: This platform provides a centralized repository for managing and accessing data assets.
  • Implement automated SQL generation: Create tools and processes to automatically generate SQL queries based on user-defined criteria or business requirements. This can involve using techniques like query templates, metadata-driven generation, or machine learning algorithms.
  • Create new IPs in Singapore in area of expertise
  • Manage collaboration projects with Institute of Higher Learning or Research Institutes on new technology development
  • Overseas process development support for short term assignments


Key Qualifications/Requirements:

  • Bachelors (w/ min. 3 yrs of relevant working exp.)/ Masters (w/ min. 2 yrs of relevant working exp./ PhD (w/ min. 1 yr of relevant working exp.
    • Computer Science,
    • Data Science,
    • Statistics Applied Sciences, or related field.


Work experience/Skillset:

  • Programming skills : Proficiency in programming like Python, SQL, Java, or Scala is essential for data manipulation and automation.
  • Data Warehousing and ETL: Understanding of data warehousing concepts, ETL (Extract, Transform, Load) processes, and tools like Informatica, Talend, or Airflow.
  • Database Management: Knowledge of relational databases (MySQL, PostgreSQL, Oracle) and NoSQL databases (MongoDB, Cassandra) is crucial.
  • Cloud Platforms: Experience with deploying and managing data infrastructure.
  • Data Modelling: Ability to design and implement data models that effectively represent business requirements.
  • Data Quality: Understanding of data quality concepts, validation techniques, and tools.
  • Automation Tools: Familiarity with tools like Apache Airflow, Kubernetes, or Ansible for automating data pipelines and infrastructure.
  • Machine Learning: Basic understanding of machine learning algorithms can be beneficial for certain automation tasks, such as query optimization or anomaly detection.