Data Engineer

Location Sydney, New South Wales, Australia
Discipline Data, BI and Analytics
Job reference 110149
Salary Negotiable

To ensure success in the role you will possess the following skills -

  • Minimum of 5+ years of experience as a solution designer, developer, tester or support role in IT Consulting or other technology business units across the industry
  • Minimum of 3+ years of hands-on development, testing, support and administration experience Apache Flink or Akka(Actors) solutions written in Scala or Java
  • Experience in applying industry best practices, design patterns and first-of-a-kind technical solutions as developer or administrator on data and non-data specific platforms and applications
  • Experience in implementing near real-time data flows/pipelines and distributed streaming applications
  • Experience in implementing traditional ETL, data warehousing and BI solutions
  • Hands-on development experience and working skill level in Python, Scalar, SQL, shell scripting and other programming/scripting languages
  • Hands-on implementation of services sensitive to performance SLAs with high availability, fault-tolerance, automatic fail-over and geographical redundancy
  • Working knowledge and hands-on experience with Cloud Platform such as Azure, AWS, GCP and IBM and other modern data platforms
  • Solid understanding of containerisation (Docker, Kubernetes etc), virtualisation, infrastructure and networking
  • Diverse experience in software development methodologies and project delivery frameworks such as Agile Sprints, Kanban, Waterfall etc
  • Degree in Computer Science, Information Technology or related Engineering courses
  • One or more industry-recognised technology certifications or badges

The Data Engineer is responsible for building and supporting the big data platform. As a Data Engineer - Big Data, your responsibilities include -

  • Develop, test and support future-ready data solutions for customers across industry verticals
  • Develop, test and support end-to-end batch and near real-time data flows/pipelines
  • Demonstrate understanding in data architectures, modern data platforms, big data, ML/AI, analytics, cloud platforms, data governance and information management and associated technologies
  • Develop and demonstrate Proof of Concepts and Working Demos
  • Support and collaborate with other internal/external consultants in consulting, workshops and delivery engagements
  • Mentor junior IBM consultants in the practice and delivery engagements