Feature Engineering Saved My Project

Does anyone ever feel like feature engineering should have hero status in the data science world?

I certainly do.

Because late last week, I ran into a big snag that I thought was going to derail my project. But then something unexpected happened: feature engineering saved the day.

Let’s Go to that Moment

Recently I started working on a machine learning project where I’m gathering data from several years’ worth of files.

The first few documents that I worked with were data-rich and things were going well.

But as I kept going, I realized that while some files had quite a bit of useful information, most of them didn’t.

My heart sank as I started to think that my project wasn’t going to make it past data gathering stage.

Then I Realized Something

Since all the files had a handful of shared information, that could be the baseline and I could use feature engineering to pull in outside information.

And just like that, I could keep going.

Long-Live Feature Engineering

Feature engineering’s let me pull together really interesting data from sources like the World Bank, UN and Transparency International.

And what’s even better is that those features are adding value in ways that I didn’t expect, which has opened up the project to a wider audience.

So If You Hit a Similar Snag

Remember that with feature engineering and some creativity, you can find a superhero to save the day (or your project, anyway).