Sounds interesting! I'm trying to train a model but it's still "processing" after a bit but fine-tuning takes a while I get it. I'm having trouble understanding how it's inferring schema. I used a sample dataset and yet the sample inference curl uses a blank json?
How do I know what the inputs/outputs are for one of my models? I see I could have set the response variable manually before training but I was hoping the auto-infer would work.
Separately it'd be ideal if when I ask for models that you seem to not be able to train (I asked for an embedding model as a test) the platform would tell me it couldn't do that instead of making me choose a dataset that isn't anything to do with what I asked for.
All in all, super cool space, I can't wait to see more!
I'm a former YC founder turned investor living in Dogpatch. I'd love to chat more if you're down!
1. Depending on your dataset the training could take from 45 mins to a few hours. We do need add an ETA on the build in the UI.
2. The input schema is inferred towards the end of the model building process, not right at the start. This is because the final schema depends on the decisions made regarding input features, model architecture etc during the building process. You should see the sample curl update soon, with actual input fields.
3. Great point about upfront rejecting builds for types of models we don't yet support. We'll be sure to add this soon!
We're in London at the moment, but we'd love to connect with you and/or meet in person next time we're in SF - drop us a note on LinkedIn or something :)
Thanks for the great feedback! We've added a `baseline_deployed` status where the agents create an initial baseline and deploy it so you have something to play around with quickly. This is why you're seeing a blank json there. Once your final model is deployed, it creates an input and output schema from the features used for the model build :)
Few questions:
1. Can it work with tabular data, images, text and audio?
2. Data preprocessing code is deployed with the model?
3. Have you tested use cases when ML model was not needed? For example, you can simply go with average. I'm curious if agent can propose not to use ML in such case.
4. Do you have agent for model interpretation?
5. Are you using generic LLM or have your own LLM tuned on ML tasks?
1. Tabular data only, for now. Text/images also work if they're in a table, but unfortunately not unstructured text or folders of loose image files. Full support for images, video, audio etc coming sometime in the near future.
2. Input pre-processing is deployed in the model endpoint to ensure feature engineering is applied consistently across training and inference. Once a model is built, you can see the inference code in the UI and you'll notice the pre-processing code mirrors the feature engineering code. If you meant something like deploying scheduled batch jobs for feature processing, we don't support that yet, but it's in our plans!
3. The agent isn't explicitly instructed to "push back" on using ML, but it is instructed to develop a predictor that is as simple and lightweight as possible, including simple baseline heuristics (average, most popular class, etc). Whatever performs best on the test set is selected as the final predictor, and this could just be the baseline heuristic, if none of the models outperform it. I like the idea of explicitly pushing back on developing a model if the use case clearly doesn't call for it!
4. Yes, we have a model evaluator agent that runs an extensive battery of tests on the final model to understand things like robustness to missing data, feature importance, biases, etc. You can find all the info in the "Evaluations" tab of a built model. I'm guessing this is close to what you meant by "model interpretation"?
5. A mix of generic and fine-tuned, and we're actively experimenting with the best models to power each of the agents in the workflow. Unsurprisingly, our experience has been that Anthropic's models (Sonnet 4.5 and Haiku 4.5) are best at the "coding-heavy" tasks like writing a model's training code, while OpenAI's models seem to work better at more "analytical" tasks like reviewing results for logical correctness and writing concise data analysis scripts. Fine-tuning for our specific tasks is, however, an important part of our implementation strategy.
Thanks a lot! On a side note: big fan of mljar here. When we were initially playing around with using agents for automating ML tasks, we had used problems from the openml's automl benchmark which you had posted about on Reddit for our initial tests
Product seems cool. But can you help me understand if what you are doing is different from the following:
> you put a prompt
> Plexe glorifies that prompt into a bigger prompt with more specific instructions (augmented by schema definitions, intent and whatnot)
> plug it into the provided model/LLM
> .predict() gives me the output (which was heavily guardrailed by the glorified prompt in the step 2)
Great question, and yes, it's quite different: Plexe generates code for a pipeline that processes your dataset (analysis, feature engineering, etc) and trains a custom ML model for your use case. When you call `.predict()`, it is that trained custom model that provides the response, not an LLM. The model is also hosted for you, and Plexe takes care of MLOps things like letting you retrain the model on new data, evaluating the model performance for you, etc. Using custom specialised models is generally more effective, faster and cheaper compared to running your predictions through an LLM when you have a lot of data specific to your business.
Thanks a lot! We’ve tried to mimic the workflow of an ML engineer and built agents that can own specific functions of the workflow. Good to hear that the idea resonates with you!
Great product and congratulations on the launch. Who is the target user vs customer? On the surface, and I may be wrong here, this feels like a LLM layered on top of a typical AutoML structure eg: TPOT, Caret. Is that the correct mental model for a tool like this? And if so, do you see a similar problem that these tools faced in broader adoption at companies?
I think "agents layered on top of AutoML" is a reasonable simple mental model for Plexe's model building capabilities, but it also masks some important qualitative differences between Plexe and traditional AutoML tools:
1. AutoML tools work on clean data. Data preparation requires an understanding of business context, the ability to reason on the data in that context, and then produce code for the required data transformations. Given that this process could not be automated with "templated" pipelines, teams using AutoML still have to do the hardest - and arguably most important - part of the data science job themselves.
2. AutoML tools use "templated" models for regression, classification, etc, which may not result in as good a "task-data-model fit" as the sort of purpose-written ML code a data scientist or ML engineer might produce.
3. AutoML tools still require a working understanding of data science technicalities. They automate the running of ML training experiments, but not the task of deciding what to do in the first place, or the task of understanding whether what was done actually fits the task.
With this in mind, we've seen that most ML teams don't find traditional AutoML tools useful (they only automate the "easy" part), while software teams don't find them accessible (data science knowledge is still required).
Plexe addresses both of these issues: the agents' reasoning capabilities enable it to work with messy data (as long as you provide business context), and to ENTIRELY abstract the deeper technicalities of building custom models fitting the task and the data. We believe this makes Plexe both useful to ML teams and accessible to non-ML teams.
Does this line up with your experience of AutoML tools?
In the demo, you didn’t show the process of cleaning and labeling data, does your product do that somehow, or do you still expect the user to provide that after connecting the data source.
Great question, this is super important. The agents in the platform have the ability to do some degree of cleaning on your data when building a model (for example, imputing missing values). However, major improvements to data quality are generally not possible without an understanding of the data domain (i.e. business context), so you'll get better results if you "help" the platform by providing data in a reasonably clean state, answering the agent's follow-up questions in the chat, etc. By doing so you can give the agent better context and help it understand your data better, in which case it will also be more capable of dealing with things like missing values, misnamed columns etc.
This also highlights the important role of the user as a (potentially non-technical) domain expert. Hope that makes sense!
We have a data enricher feature (still in a beta mode) which uses LLMs to generate labels for your data. For cleaning and feature engineering, we use agents that automatically handle it for you once you've connected your data and defined your ML problem.
P.S. Thanks for the feedback on the video! We'll update it to show the cleaning and labelling process :)
Thanks for your feedback! The "export analysis" functionality is built to enable you to get detailed data insights and get a report generated from the insights. Would you prefer to see the entire chat in your export or would it be helpful for it to simply include code snippets in the same format you received right now?
A way to select all (or by Q/A paragraph) for export (e.g. to pdf or how about a .ipynb) would probably be the most useful. Perhaps as just a plain "export" option in addition to an analysis/insights summary.
p.s. kudos on the promo code that enable folks to kick the tires with as little friction as possible.
very cool – I like how opinionated the product approach is vs. a bunch of disconnected tools for specialists to use (which seems more common for this space).
Thanks, we're pretty opinionated on "this should make sense to non-ML practitioners" being a defining aspect of the product vision. Behind the scenes, we've had quite a few conversations specifically about how to avoid features feeling "disconnected", which is always challenging at an early stage when you're getting pulled in several directions by users with different use cases. Happy to hear it came across that way to you.
Unfortunately we don't officially support image, video or audio yet - only tabular data for now. We do plan to add that capability at some point in the coming weeks depending on popular demand. Do you have any particular use case in mind?
Caveat: as a more technical user, you can currently "hack" around this limitation by storing your images as byte arrays in a parquet file, in which case the platform can ingest your data and train a CV model for you. We haven't tested the performance extensively though, so your mileage may vary here.
Sounds interesting! I'm trying to train a model but it's still "processing" after a bit but fine-tuning takes a while I get it. I'm having trouble understanding how it's inferring schema. I used a sample dataset and yet the sample inference curl uses a blank json?
curl -X POST "XXX/infer" \ -H "Content-Type: application/json" \ -H "x-api-key: YOUR_API_KEY" \ -d '{}'
How do I know what the inputs/outputs are for one of my models? I see I could have set the response variable manually before training but I was hoping the auto-infer would work.
Separately it'd be ideal if when I ask for models that you seem to not be able to train (I asked for an embedding model as a test) the platform would tell me it couldn't do that instead of making me choose a dataset that isn't anything to do with what I asked for.
All in all, super cool space, I can't wait to see more!
I'm a former YC founder turned investor living in Dogpatch. I'd love to chat more if you're down!
Thanks for the great feedback! To your points:
1. Depending on your dataset the training could take from 45 mins to a few hours. We do need add an ETA on the build in the UI.
2. The input schema is inferred towards the end of the model building process, not right at the start. This is because the final schema depends on the decisions made regarding input features, model architecture etc during the building process. You should see the sample curl update soon, with actual input fields.
3. Great point about upfront rejecting builds for types of models we don't yet support. We'll be sure to add this soon!
We're in London at the moment, but we'd love to connect with you and/or meet in person next time we're in SF - drop us a note on LinkedIn or something :)
Thanks for the great feedback! We've added a `baseline_deployed` status where the agents create an initial baseline and deploy it so you have something to play around with quickly. This is why you're seeing a blank json there. Once your final model is deployed, it creates an input and output schema from the features used for the model build :)
Amazing! Great work. Congratulations on launch.
Few questions: 1. Can it work with tabular data, images, text and audio? 2. Data preprocessing code is deployed with the model? 3. Have you tested use cases when ML model was not needed? For example, you can simply go with average. I'm curious if agent can propose not to use ML in such case. 4. Do you have agent for model interpretation? 5. Are you using generic LLM or have your own LLM tuned on ML tasks?
Thanks! Great set of questions:
1. Tabular data only, for now. Text/images also work if they're in a table, but unfortunately not unstructured text or folders of loose image files. Full support for images, video, audio etc coming sometime in the near future.
2. Input pre-processing is deployed in the model endpoint to ensure feature engineering is applied consistently across training and inference. Once a model is built, you can see the inference code in the UI and you'll notice the pre-processing code mirrors the feature engineering code. If you meant something like deploying scheduled batch jobs for feature processing, we don't support that yet, but it's in our plans!
3. The agent isn't explicitly instructed to "push back" on using ML, but it is instructed to develop a predictor that is as simple and lightweight as possible, including simple baseline heuristics (average, most popular class, etc). Whatever performs best on the test set is selected as the final predictor, and this could just be the baseline heuristic, if none of the models outperform it. I like the idea of explicitly pushing back on developing a model if the use case clearly doesn't call for it!
4. Yes, we have a model evaluator agent that runs an extensive battery of tests on the final model to understand things like robustness to missing data, feature importance, biases, etc. You can find all the info in the "Evaluations" tab of a built model. I'm guessing this is close to what you meant by "model interpretation"?
5. A mix of generic and fine-tuned, and we're actively experimenting with the best models to power each of the agents in the workflow. Unsurprisingly, our experience has been that Anthropic's models (Sonnet 4.5 and Haiku 4.5) are best at the "coding-heavy" tasks like writing a model's training code, while OpenAI's models seem to work better at more "analytical" tasks like reviewing results for logical correctness and writing concise data analysis scripts. Fine-tuning for our specific tasks is, however, an important part of our implementation strategy.
Hope this covers all your questions!
Thanks a lot! On a side note: big fan of mljar here. When we were initially playing around with using agents for automating ML tasks, we had used problems from the openml's automl benchmark which you had posted about on Reddit for our initial tests
Product seems cool. But can you help me understand if what you are doing is different from the following: > you put a prompt > Plexe glorifies that prompt into a bigger prompt with more specific instructions (augmented by schema definitions, intent and whatnot) > plug it into the provided model/LLM > .predict() gives me the output (which was heavily guardrailed by the glorified prompt in the step 2)
Great question, and yes, it's quite different: Plexe generates code for a pipeline that processes your dataset (analysis, feature engineering, etc) and trains a custom ML model for your use case. When you call `.predict()`, it is that trained custom model that provides the response, not an LLM. The model is also hosted for you, and Plexe takes care of MLOps things like letting you retrain the model on new data, evaluating the model performance for you, etc. Using custom specialised models is generally more effective, faster and cheaper compared to running your predictions through an LLM when you have a lot of data specific to your business.
> Each agent focuses on its domain: data analysis, feature engineering, model selection, deployment, and so on
Sounds very practical in real-world use cases. I trained a ML model couple months ago, I think it's a good case to test this product.
Thanks a lot! We’ve tried to mimic the workflow of an ML engineer and built agents that can own specific functions of the workflow. Good to hear that the idea resonates with you!
Great product and congratulations on the launch. Who is the target user vs customer? On the surface, and I may be wrong here, this feels like a LLM layered on top of a typical AutoML structure eg: TPOT, Caret. Is that the correct mental model for a tool like this? And if so, do you see a similar problem that these tools faced in broader adoption at companies?
I think "agents layered on top of AutoML" is a reasonable simple mental model for Plexe's model building capabilities, but it also masks some important qualitative differences between Plexe and traditional AutoML tools:
1. AutoML tools work on clean data. Data preparation requires an understanding of business context, the ability to reason on the data in that context, and then produce code for the required data transformations. Given that this process could not be automated with "templated" pipelines, teams using AutoML still have to do the hardest - and arguably most important - part of the data science job themselves.
2. AutoML tools use "templated" models for regression, classification, etc, which may not result in as good a "task-data-model fit" as the sort of purpose-written ML code a data scientist or ML engineer might produce.
3. AutoML tools still require a working understanding of data science technicalities. They automate the running of ML training experiments, but not the task of deciding what to do in the first place, or the task of understanding whether what was done actually fits the task.
With this in mind, we've seen that most ML teams don't find traditional AutoML tools useful (they only automate the "easy" part), while software teams don't find them accessible (data science knowledge is still required).
Plexe addresses both of these issues: the agents' reasoning capabilities enable it to work with messy data (as long as you provide business context), and to ENTIRELY abstract the deeper technicalities of building custom models fitting the task and the data. We believe this makes Plexe both useful to ML teams and accessible to non-ML teams.
Does this line up with your experience of AutoML tools?
In the demo, you didn’t show the process of cleaning and labeling data, does your product do that somehow, or do you still expect the user to provide that after connecting the data source.
Great question, this is super important. The agents in the platform have the ability to do some degree of cleaning on your data when building a model (for example, imputing missing values). However, major improvements to data quality are generally not possible without an understanding of the data domain (i.e. business context), so you'll get better results if you "help" the platform by providing data in a reasonably clean state, answering the agent's follow-up questions in the chat, etc. By doing so you can give the agent better context and help it understand your data better, in which case it will also be more capable of dealing with things like missing values, misnamed columns etc.
This also highlights the important role of the user as a (potentially non-technical) domain expert. Hope that makes sense!
We have a data enricher feature (still in a beta mode) which uses LLMs to generate labels for your data. For cleaning and feature engineering, we use agents that automatically handle it for you once you've connected your data and defined your ML problem.
P.S. Thanks for the feedback on the video! We'll update it to show the cleaning and labelling process :)
Problem space is very interesting. Sounds like most of the work will be the data handling which is an evergreen problem
Absolutely! Getting AI agents to determine the right set of features from raw data has been a very interesting problem
The tool gave me advice and code to do what I asked... but when I used the "export analysis" it did NOT include the code. It was simply an overview.
It would be more useful for the export to have an option (or by default) to include everything from the session.
Thanks for your feedback! The "export analysis" functionality is built to enable you to get detailed data insights and get a report generated from the insights. Would you prefer to see the entire chat in your export or would it be helpful for it to simply include code snippets in the same format you received right now?
A way to select all (or by Q/A paragraph) for export (e.g. to pdf or how about a .ipynb) would probably be the most useful. Perhaps as just a plain "export" option in addition to an analysis/insights summary.
p.s. kudos on the promo code that enable folks to kick the tires with as little friction as possible.
Makes sense! We'll be sure to make this available very soon :) Thank you!
Really diggin this. Can't wait to try it out.
Thanks a lot! Excited for you to try it out and get your feedback :)
Interesting. Are you guys only charging for the model building using tokens or overall usage (model building, inferences, etc)?
It's a combination of tokens consumed, dataset + model storage cost, and inference + training compute cost.
very cool – I like how opinionated the product approach is vs. a bunch of disconnected tools for specialists to use (which seems more common for this space).
Thanks, we're pretty opinionated on "this should make sense to non-ML practitioners" being a defining aspect of the product vision. Behind the scenes, we've had quite a few conversations specifically about how to avoid features feeling "disconnected", which is always challenging at an early stage when you're getting pulled in several directions by users with different use cases. Happy to hear it came across that way to you.
How does it perform when build computer vision models?
Unfortunately we don't officially support image, video or audio yet - only tabular data for now. We do plan to add that capability at some point in the coming weeks depending on popular demand. Do you have any particular use case in mind?
Caveat: as a more technical user, you can currently "hack" around this limitation by storing your images as byte arrays in a parquet file, in which case the platform can ingest your data and train a CV model for you. We haven't tested the performance extensively though, so your mileage may vary here.
Great product!
Thank you! :)