As a data engineer dedicated to optimising workflow and automating mundane tasks, I rely on a suite of software and hardware to maximise productivity on my 2021 M1 Pro MacBook. In this introductory post, I'll share my indispensable tools for seamless data engineering work.
Software Essentials:
Raycast serves as an efficient spotlight replacement, allowing me to manage windows, use the calculator feature, and access clipboard history swiftly. In a forthcoming blog post, I'll delve deeper into my Raycast setup and daily-use extensions.
Development Environments
- Visual Studio Code
- JetBrains Toolbox (DataGrip, PyCharm, IntelliJ, Rider)
- Vivaldi (for browsing efficiency)
- CyberDuck (for file transfers)
- Postman (API testing)
- Shottr (for screen capturing)
- Amplenote (note-taking)
Terminal
I rely on iTerm2 with oh-my-ZSH installed for an enhanced terminal experience. This setup streamlines my coding and scripting tasks.
Version Control & Collaboration
- GitKraken or Github Desktop or GitButler
- Slack (for team communication)
Entertainment and Learning
- Spotify (music for focus and podcast listening)
- Blogs: Regularly follow tech blogs like Uber, Airbnb, Clickhouse, StartDataEngineering, and N8N blog for valuable insights.
Online Communities
- Engage with active and informative subreddits like r/SideProject, r/DataEngineering, and r/DataIsBeautiful.
Self-hosting on VMs
I manage two Hetzner VMs (CX21) to run multiple services efficiently. My essential self-hosted tools include:
- Coolify: An open-source alternative to Heroku/Netlify/Vercel, enabling seamless hosting of various services.
- NiFi: An open-source for data flow builder
- Clickhouse: An open-source OLAP database
- Ghost: Hosting platform for this blog.
- N8N: A powerful workflow automation tool.
- NocoDB: An open-source Airtable alternative.
- Directus: An open-source CMS for content management.
- UptimeKuma: Service monitoring tool.
- PostgreSQL DBs: For side projects and data collection.
Implementing these tools has significantly boosted my productivity and allowed me to streamline data engineering tasks on my MacBook.
Stay tuned for more in-depth discussions on optimising these tools and their configurations in upcoming blog posts!