How are you operating AI infrastructure in production?

A Hacker News user asked the community how they operate open-source AI infrastructure in production, seeking recommendations on tools for inference, orchestration, observability, vector search, data pipelines, evaluation, and model management, as well as insights on self-hosting versus managed services and challenges beyond prototyping. The post received 1 point and no comments.

There are many open-source projects across inference, orchestration, observability, vector search, data pipelines, evaluation, and model management. Most are relatively easy to test, but production operation is a different problem. For those running open-source AI infrastructure in production: - What are you running, for what workload, and would you recommend? - Do you operate yourself versus consume as a managed service? - Have you replaced or abandoned any tools because they were too difficult or expensive to operate? - What problems only appeared after moving beyond the prototype stage? - Anything that you would do differently if rebuilding the stack today? Thanks Comments URL: https://news.ycombinator.com/item?id=49163280 https://news.ycombinator.com/item?id=49163280 Points: 1 Comments: 0