Data Engineer
About Bigspark
bigspark, a UK based consultancy delivering next level data platforms and solutions with a focus on exciting technologies including Apache Spark, Apache Kafka and working on projects within Machine Learning, Data Engineering, Streaming and Data Science is looking for a Data Engineer to join our team on a permanent basis in a UK remote, work from home capacity with some attendance to client sites where necessary.
We provide the backbone for modern analytics to our clients through expertise in DevOps, distributed computing, machine learning and adoption of proven open source projects. We specialise in backend development, infrastructure automation and performance engineering for data workloads at scale.
Your Role
In your hands-on role of Data Engineer you will
- Adhere and contribute to our delivery principles
- Develop and support scalable, extensible, and highly available data pipelines on heterogeneous datasets that power downstream applications
- Join a team with skills in either Python, Java, Scala in a remote, work from home capacity with some attendance to client sites, where necessary
- Interact with stakeholders and program management to understand client requirements
- Solution Design – Using predefined architectural components to agree an acceptable design
- Deliver software development on client site, whilst collaborating with team members at various levels across the globe
- Warranty support post-delivery (normally 1-2 months)
Your Skills, Experience and Qualifications
- Educated to at least BSc (Hons) level – preferably in Computer Science, Data Science or Mathematics
- At least 3-5 years in a commercial software environment
- Experience delivering multiple solutions using key techniques such as Data Modelling, DWH, Lakes, ETL, ELT, Virtualisation & Streaming
- Big data stack experience in one or more of the tools – Hadoop, Spark, Hive/Impala, Kafka, Snowflake
- Experience of storage formats – Avro, Parquet, ORC
- Proficiency in object-oriented programming languages (Python, C#, Java, Scala)
- Experience with database integrations (SQL/NoSQL) and processing data at scale
- Understanding of source control management e.g. GitHub
- Experience in use of CI tools (Jenkins) or an understanding of their role
- Flexible in learning and working with new technologies
How we help you succeed
- Remote, work from home position
- Competitive salary
- Discretionary Annual Bonus
- Generous Annual Leave (25 Days + 8 Bank Holidays)
- Pension Scheme
- Life Assurance
- Private Medical Cover (inc family)
- Permanent Health Insurance Cover / Income Protection
- Employee Assistance Programme
- A Perkbox account
- Access to an Independent Financial Advisor
- Top end equipment
