Technology


Credibility is key to any architect or data leader. This credibility needs to be earned by gaining knowledge of technology not by reading a sales pitch but by experiencing the technology. I often find myself lost in my day-to-day deliverables and have difficulty keeping up with evolving tools, especially as data technologies evolve rapidly. Sure, I can look at a demo, but I can truly understand something only when I touch, feel, and experience it. Being able to quickly prototype and test solutions goes a long way in building a concept which I can then build upon and conceptualize how I can fit it into a larger architecture.

Hands-on knowledge of technology is important to me to ensure that I can design practical, scalable, and efficient data architectures. It enhances my ability to communicate with technical teams, ensures the quality and performance of solutions, and fosters continuous improvement and innovation. It makes it easier to identify potential issues and bottlenecks in the architecture early in the design and implementation phases.

I have seen so many bad decisions made because leaders do not understand the technology constraints and opportunities. By setting up a sandbox and piloting a quick demo to highlight the capabilities of a technology and how they align with the data architecture and strategy, we can better translate business requirements into technical solutions that are realistic and achievable.

From personal experience, hands-on knowledge allows me to build rapport with data engineers and developers who I have to lead. They are more likely to trust and follow the guidance of someone who understands the intricacies of the work. At the same time, it also helps to understand complexity and review the true progress of the development team.

AWS Lake Formation.

Data Lake, LakeFormation, 2025

AWS Lake Formation = Scaled Data Lake + Scaled Security Provisioning image

ETL Data Test and Validate

Data Warehouse, Snowflake, 2024

Setup

  1. Installed DBT Core locally. The install configuration can be described by the command: dbt debug image

  2. Installed DBT Utils image

    image

MinIO Object Storage for Linux Locally (anywhere)

Lakehouse, Minio, 2024

When I play with new technologies, I like to plat it on my machine locally. Minio is a perfect simulation of cloud storage locally. You can deploy it locally and interact it like a S3 object storage.