‘Not every data engineer gets client experience like this’

By HANNAH MARTIN

Covering everything from system design to German corporate hierarchies, Hong’s first couple of years at idalab have been full of challenge and opportunity – which is just how she likes it 

Why did you join idalab?

My story began with an internship. I had taken some machine learning courses at university and been involved in data mining and analysis in the research lab – but I was curious about the world outside. I wanted to know whether these sophisticated theories were accepted and applied by the industry – and, if so, to what extent? idalab seemed to be a perfect choice for me: a professional team, interesting client projects and, more importantly, a growth mindset that’s constantly encouraged.

So I joined idalab as an intern for data engineering. My role wasn’t exactly what I was expecting, at least in the first three months, as there was quite a lot of system design and computer platform improvement before I finally got to start “working with data”. At that time, we were in a transition regarding our computing platforms: we wanted to move from traditional on-premises servers to more professional and modern cloud solutions. This motivated my first project at idalab: deploying Kubeflow, an end-to-end machine learning platform for Kubernetes, in a cloud environment. On top of ensuring it integrated with idalab’s existing setups, it also had to be compatible with future extensions.

Hong enjoys the idalab ethos: ‘We have the kind of team that always surprises you, in a very cute way’

I must say that I felt very lucky at idalab to have such supportive colleagues, as well as a very experienced mentor. Container orchestration was not my most familiar area, but they helped me learn it systematically and get hands-on experience. Solid computing architecture is the cornerstone of data engineering, so I’m glad this is where I started.

Which project challenged you the most and why?

Each project has a specific context, and so each has its own challenges, especially when it comes to client projects. In one of my recent projects, with a client from the public sector, we used natural language processing and deep learning to help them automate time-consuming and quality-sensitive data-curation tasks. The client is a typical German organisation with a complex organisational structure; navigating this type of difficult stakeholder situation is challenging for a fledgling student – especially one from a different cultural background.

But this is also the fun part! You won’t get this sort of practice and experience everywhere. Thanks to Paul and Benjamin, two colleagues who are very experienced with client projects, I received a lot of practical suggestions. We have our regular debrief sessions, where we study which parts we’ve done well and which should be improved, and how to improve them. Before important client Jour-Fixes we also run mock meetings to prepare. These practice sessions have really helped me to get into the role.

This project is also technically quite challenging; it’s my first machine learning project in production, and some special requirements require careful consideration. It is very special for me because I had the opportunity to build direct contact with the clients as well as to improve my skills of stakeholder management and communication.

This is not something all data engineers get a chance to do, and I enjoyed the process a lot. I think my German has even improved – to the extent that things like “Produktivsetzung” or “Verfahrensrollenbeschreibungskonzept” are now second nature to me!

What is your current project and what is your role on it?

I’m now working on improving of our internal T-Modules (Technical Modules), which are idalab’s own technical training frameworks. Ever since I joined, I’ve really enjoyed helping colleagues who maybe don’t have so much technical knowledge. But I can also draw on my co-workers’ knowledge of  topics I don’t know so much about. I think this is a very beautiful part of team collaboration. 

With Luis and Jeanette: ‘I can always draw on my co-workers’ knowledge. This team collaboration is beautiful’

It turns out that many problems we encounter during data manipulation and analysis appear repeatedly. For instance, I was once asked by three colleagues about  “virtual environments” almost simultaneously. I also remember a database modelling problem a colleague asked me about coming up again half a year later. So we came up with an idea: why not integrate these topics into a knowledge base for internal consumption? It would be a bit  like Wikipedia – a collaborative knowledge platform where everyone is encouraged to share their knowledge and interest. 

This was the motivation behind our T-Modules. The one I’m currently working on is the third part of it, about modern database systems. At the same time, I’m also helping develop the online test and challenge for our data engineer recruiting process. We would like to continue growing the engineering team, and a revised version of recruitment tests – based on the actual working requirements – would really help.

What does a typical day look like for you?

This is a fairly typical working day for me. If I’ve got no meetings, I’ll use the morning time to reply to emails, and check out work progress and schedules. On this particular day, Benjamin and I had a Jour-Fixe with the clients to discuss the current cloud infrastructure setups. We usually have a preparation meeting beforehand and a short debrief afterwards. 

Later in the day I had another Jour-Fixe for an internal technical training project with Rouven and Raphael. Then I had a feedback session with Luis, in which  he gave us some feedback about the new recruiting online test we were working on. Luis and I had another one-to-one meeting afterwards; I forgot to mention that Luis is my mentor. In these meetings we talk about the progress of my recent work, my study, how my cloud training course is going … or anything else that he could help me with. These one-to-one conversations, which we have once a month, are very relaxing and enjoyable. 

Before the end of each working day, I take some time to organise our cloud platform – to make sure the cluster is running a healthy status and check if there are any warnings that should be resolved or any compute/storage resources that can be recycled. 

Do you have any anecdotes you’d like to share with us?

At idalab we have the kind of team that always surprises you, in a very cute way. I’m based in Munich, as that’s where I’m studying for my master’s, so I work remotely. But I once received an invitation to an event – “Details secret,” it stated – that Juliane wanted me to attend in person. 

Hong (right) visits the office from her base in Munich all too rarely. Seen here catching up with Juliane (left)

“Sure,” I said. So I bought a ticket, booked a hotel and travelled to Berlin. Along the way I kept thinking about what kind of event it could be. There were no clues – it could be anything.

So I arrived at the office, met my lovely colleagues, enjoyed a coffee break with them, and even had two meetings (finally in-person) before the start. By the time the event started, the whole Arena was beautifully decorated – I don’t know when that happened. Then Paul started to read my name, someone handed me a glass of champagne, and I was told I was being promoted, from student trainee to junior associate. Happiness comes so suddenly! This is so idalab. 

Have you always worked in the position you now have at idalab?

Not at all. My role has changed a lot, both in terms of working content and target clients. As I mentioned before, I was mainly working on internal infrastructure for my first three months. My role at this stage was more about how to convince colleagues to accept the new infrastructure. I’d also listen to their suggestions and do modifications, as well as arrange follow-up coaching sessions. 

After that, I started to gradually get involved in client projects. We wanted to build on the success of our internal infrastructure by looking for ways it could be beneficial to client projects. This gave me a chance to apply existing internal experiences in a client project context. 

Right now, I’m the only one in the team who’s on the engineering program track, as we still do a lot of the engineering work with specialised partner companies. But, as I mentioned before, we’re actively preparing the recruitment process for another data engineer. I am very excited about new members joining the engineering team. We will have more capacities to enhance our internal infrastructure and explore new trends and technologies, as well as communicate about learning experiences. As for myself, in addition to continuing to explore emerging technologies, I want to focus more on enriching my domain knowledge, especially in the fields of biomedical computing and protein prediction.

Contact

Hannah Martin

Associate

Mobile
+49 (0) 174 27 18 138

E-Mail
hannah.martin@idalab.de

Address
Potsdamer Straße 68
10785 Berlin