Embracing Agile: Lessons from a Data Engineering Bootcamp

01 Jun 2024

I anticipated that the Data Engineering Bootcamp at Northcoders would be fast-paced and challenging. What I didn’t expect was the intense and rewarding social experience of working together with other trainee data engineers to learn, solve problems, and deliver a complex and polished product in the final three weeks.

I had heard of pair programming, agile, and scrum, and I’d even read up on them and tried to incorporate some of the principles of agile into my work as an architect. But there’s nothing that compares to rolling up your sleeves and getting stuck in with a fellow coder, sharing challenges and celebrating breakthroughs. This is all old news for anyone already working in the tech industry, but I wanted to record my enjoyment of the process at the start of my journey in a new industry.

In completing our final ETL pipeline project, here are a few of the key elements that I think helped us succeed:

• Questioning and clarifying the brief at the outset. As an architect, collaboratively devloping, refining, and crystalising a brief comes naturally, and I love the proccess of helping a client or stakeholder really question their own needs and motivations to demand the best product.
• Clarity in system architecture and interfaces early on Each element of the system was described in great detail before writing any code. Their inputs, outputs, possible errors, data types and formats, payloads. Individual functions were scoped and docstrings written at the outset, so when a pair picked up a ticket, they could focus on the task at hand, with confidence that if they could meet the spec, the gears would mesh and the system would work.
• Flexibility in teams and reactivity to blockers Regular communication and collaboration were crucial in overcoming obstacles. When significant blockers, bugs, or technical hitches arose, the entire team could pitch in, ensuring that development progress was never stalled. This agility and responsiveness were instrumental in maintaining momentum and delivering the project on time.

As I transition into the tech industry, I am excited to dive deeper into this collaborative and agile way of working. The Data Engineering Bootcamp has not just imparted technical skills but has also instilled in me a profound appreciation for the power of teamwork, communication, and adaptability in software development.

Wedding Playlist

27 Jul 2023

Click here for our wedding playlist on Spotify

VCV Rack

05 Dec 2019

VCV rack - the wonderful absurdity of simulating analog signal processing using the power and speed of a modern computer. The joy is akin to playing the Zachtronics games TIS-100 or Shenzen/IO. Begin with the flexibility of a powerful all-purpose computing platform, overlay a veneer of simplicity and constraint, then allow open experiment. Complexity and texture and life start to emerge.

The Phonetics of Gregg Shorthand

30 Nov 2019

I started getting into hand stenography after learning Plover. What I looked for in a writing system mirrored what I loved about machine steno:

• As phonetic as possible
• Consistent and predictable to learn
• Fast

I also added criteria:

• Beautiful
• Fluid

Gregg shorthand met all of these, and more. The choice of curves and lines follows a host of predictable phonetic rules and (at least to me) seems to even echo the physiology of producing the respective sounds. See below for some of the basic phonetic symbols. The line shape mimics the tongue position and the length indicates voicing (short for unvoiced, long for voiced).

Letter Sound	Tongue Position	Voiced	Symbol
R	short under-curve, sides touching palette	yes
L	long under-curve, tip touching palette	yes
K	arch to pallete	no
G	arch to pallete	yes
T	diagonal up to teeth	no
D	diagonal up to teeth	yes

Delight

25 Oct 2019

With the release of iOS 13 I have enjoyed exploring the tweaks and changes in some of the most frequently used interactions. The volume indicator is particularly delightful. It is grabbable, grows when grabbed, has an informative animated icon and haptic stops top and bottom. It also pops on to the screen perfectly aligned with the hardware volume buttons. It is as if the physical act of pressing the buttons pushes the virtual slider onto the screen. This is Apple’s famous hardware/software integration distilled into a tiny control. It may seem overkill, but I use this control hundreds of times per day, and now wonder how it could ever have been otherwise.