About

My name is Daniel Lavedonio de Lima and I’m curious by nature. I’m an electrical engineer working with data, and I’ve always been excited about technology and its power to change the world.

That curiosity is what brought me here: I’m always looking for new things to learn and, whenever I can, for ways to put them into practice and make a difference, even a small one. If something on this website helps you somehow, I’ll be one step closer to that.

This website is itself a result of that: a place to keep and share my projects and ideas, and a project in its own right, which taught me how the internet works from the side of who runs a website, not just who visits it.

Below you’ll find more about my education, my career, my projects and how to work with me.

I studied Electrical Engineering with an emphasis on Computing at the Polytechnic School of the University of São Paulo (Poli-USP), graduating in 2020.

In my first semester, I joined Poli Júnior, the first engineering junior enterprise in Brazil, where I worked on Production Engineering and Computer Engineering projects, in the latter also as a manager. I was also one of the organizers of the 12th Ser Empreendedor, an event that promotes entrepreneurship among university students.

In my 5th and 6th semesters, I was the Systems Coordinator at FEJESP, the Federation of Junior Enterprises of the State of São Paulo, where I was responsible for migrating its website and its e-mail system.

Towards the end of my degree, I took part in BioGadget, an undergraduate research project that used Bluetooth Low Energy (BLE) to build trackers for studies of wild animals. Although I didn’t stay until the end, it gave me the chance to get to know the academic world.

I’m currently pursuing a master’s degree at the Department of Electronic Systems Engineering of Poli-USP, researching the robustness of Natural Language Processing (NLP) systems against character-based adversarial attacks.

I started my career in data and have stayed in it ever since, going from automating spreadsheets to building data platforms in the cloud.

Marisa (December 2017 – January 2018): in my first internship, I watched analysts and interns put together the same Excel reports by hand every single day. After a talk with my boss, I spent the rest of my summer internship automating them with VBA.

Medcel (January – June 2019): besides more VBA automation, I helped the data analyst with SQL queries, Tableau dashboards and Python scripts. I also built a web scraper with Selenium to find everyone who passed medical residency exams and which of them had been our customers, in time for a board meeting two weeks later.

Mobly (July – November 2019): in a bigger team, I got to know the work of a data engineer up close, along with tools like Apache Airflow and Pentaho and, for the first time, the cloud, on AWS. That’s when I decided to move fully into data engineering.

Revelo (December 2019 – February 2021): I joined as a “senior intern” and was soon promoted to data engineer, the only one in the company. My main project took the analysts’ queries off the production database: an Airflow instance on GCP copied a replica of the Postgres database on AWS to Google Cloud Storage and loaded it into BigQuery, all built from scratch. At the end of 2020, I also took on SRE duties, automating my projects with GitHub Actions and adding observability with Prometheus and Grafana.

InfoPrice (from March 2021): in my first job after graduating, I joined a data engineering team whose data was, for the first time, the company’s main product instead of the back office. There I worked on one of the projects I’m proudest of: cutting the cost of the Spark jobs on AWS EMR by moving their machines from On-Demand to Spot instances.

Building things is one of the ways I learn best, which is why I keep personal projects going: to make my own work easier, to practice and learn new concepts, or simply because I felt like it.

The biggest one is this website. In August 2019, I decided to make an old dream come true: a place on the internet that’s truly mine, where I can share my projects and ideas without the limits of social networks. I built it from scratch with Python and Django, and it went live in April 2020. Its code is open source and this is its second version.

Another one is Pong, a recreation of the greatest video game classic, which started in an online JavaScript course and that I reworked so it can also be played on phones and other touch screens.

You’ll find them, and the stories behind them, on the Projects page and in the Blog. More of my code is on my GitHub.

I’m a data engineer: I design, build and maintain data pipelines and platforms, automate processes and make the most of cloud services. Over the years I’ve worked with Python, SQL, Apache Airflow, Spark, AWS and Google Cloud, and I also build web applications with Django, like this website.

If you have a job opportunity, a project or an idea you’d like to talk about, send me a message or reach out on LinkedIn. A few lines about the problem you want to solve and your timeline help me give you a useful answer.