Big Data Engineer

Big Data Engineer

Sciente International Pte Ltd
Kuala Lumpur
3 - 12 Years
Not Specified

Job Description

Mandatory Skill-set

  • Bachelor in IT, Computer Science or Engineering;
  • Hands on experience using Big Data technologies like Hadoop distribution (Hortonworks), Hive, HBase, Spark, Pig, Sqoop, Kafka and Spark Streaming.
  • Experience using Microsoft Azure technologies like Big Data Storage Gen2, Data Factory, Databricks, Azure ML;
  • At least 2-3 years of solid hands-on development experience with ETL tools to transform complex data structure in multiple data sources environment, such as Talend, SSIS and IBM DataStage;
  • At least 1-3 years of big data programming techniques on coding in Python, Java, R and Scala;
  • Experience in creating data model (relational and/or data warehouse), data mart design and implementation;
  • Strong knowledge in various database technologies (RDBMS, NoSQL and columnar);
  • Good understanding of data analytics and data visualization such as Qlik Sense, Tableau and Power BI;
  • Ability to communicate and present technical information in a clear and unambiguous manner;
  • Strong ability to work independently and cooperate with diverse teams in a multiple stakeholder’s environment;
  • Strong sense of work ownership, high affinity with anything data and a desire for constant improvements.
Desired Skill-set

  • Prior experience in Insurance domain.
Responsibilities

  • Design, develop, document and implement end-to-end data pipelines and data integration processes, both batch and real time;
  • Perform data analysis, data profiling, data cleansing, data lineage, data mapping and data transformation;
  • Develop ETL / ELT jobs and workflows, and deployment of data solutions;
  • Monitor, recommend, develop and implement ways to improve data quality including reliability, efficiency and cleanliness to optimize and fine tune processes;
  • Recommend, execute and deliver best practices in data management and data lifecycle processes, including modular development of data processes, coding and configuration standards, error handling and notification standards, auditing standards, and data archival standards;
  • Prepare test data, create and execute test plans, test cases and test scripts;
  • Collaborate with different stakeholders to understand data needs, gather requirements and implement data solutions to deliver business goals;
  • Provide technical support for any data issues and change requests, document all investigations, findings, recommendations and resolutions.

Job Details

Industry:

Function:

IT

Education:

Sciente International is a multi-award winning ISO9001:2008 certified Technology Recruitment Specialist firm. It is a member of Sciente Group with a mission to delight the clients and community.

Recent Recognition of Sciente International Pte Ltd:

- Won multiple awards by Singapore HR Awards 2017 organised by SHRI - Leading HR practices for CSR, Work-life Integration, Tech HR Resource Management and L&D
- Voted the Best Recruitment Firm – Engineering & IT positions by HRM Asia Readers’ Awards 2016
- Award for the Leading HR Practices in Corporate Social Responsibility by Singapore HR Awards 2016, SHRI
- Silver Asia Recruitment Award 2016 for the Best Recruitment Innovation by a Recruitment Agency
- Red Herring Top 100 Asia 2015


We provide managed services (Retained search, Rapid Team Formation & Recruitment Process Outsourcing) and contingent (Permanent & Contract) recruitment services.

Our team has decades of hands-on technology experience and industry expertise. And we operate ethically with honesty, transparency & integrity. These are some of the reasons why large multi-national companies rely on us to search the Business Technology talent and we are trusted by talented candidates to represent them.

Please visit www.scienteinternational.com to know more about us.

EA License: 07C5639

About Recruiter

232

Followers

133

Active jobs

Functions : IT

Industries : ITES/BPO, IT/Computers - Hardware, IT/Computers - Software

Skills/Roles I hire for: All

level Hiring For: Junior Level, Mid Level, High Level, Top Management

Similar Jobs

Career Advice to Find Better