Learn and Burn

I’m deep in the throes of machine learning right now. Not in the LLM sense but in the traditional classifier model sense. Not generative AI but decisioning and clustering. Old school.

I have done some of this in the past but it’s been a few years. I’m working with some Data Scientists that know the math and the models but are a little more gun shy around just hacking stuff together for some reason.

I’m thinking a little bit about what the difference is between Science and Engineering. Scientists have a certain reserve about them that I’m not used to. In any case, modeling is an art and dare I say it a Science. I’m trying to bridge the gap between scrappy startup experimentation and actual scientific experimentation. What we mean when we say “experiment” can vary pretty widely.

Anyway back to what’s at hand. I learned a few great things from the Data Scientists recently. I’m learning to embrace the Python notebook, and Hex in particular. I’ve always been a proponent of the axiom “if it’s not checked in it doesn’t exist”. This is a modern version of the “it works on my machine” rebuttal. In this case it frustrated me that they weren’t working in traditional version control like I’m used to. Maybe this is a sea change though. Sort of like how DevOps kind of ate the world. Maybe Data Science is changing Engineering too?

So to get back to the topic at hand. I’m trying to apply my Architecture training to things like whether to invest in Sagemaker or do a hand-rolled Pytorch ECS instance. None of this makes sense to me right now because the pricing is so out of whack. To me a ML model is just some code that runs in the infrastructure somewhere and spits out an answer. Do we need GPUs for this? It depends on the model. Oh man.

How often does the model need to be trained or updated? Who knows. Well the Data Scientists know. But it’s a stochastic dance around the campfire of knowledge.

I think sometimes that the folks that studied Philosophy in school and converted to Computer Science are actually the sane ones. The further along I get the more intrigued I get with Philosophy. Is it too late for me to don a toga and opine in the streets of San Francisco? Too late I’m afraid.