ability to swap meta learning algorithm with `caruana`

[frankiethull]

would it be possible to add an option to blend candidates with glmnet or caruana?

The caruana ensemble selection algorithm is a greedy iterative procedure that builds an ensemble by sequentially adding models from a pool with replacement to maximize a specific evaluation metric. This would provide an alternative to glmnet for tidymodels users.

sidenote: this is to mimic the TabArena & BeyondArea ensemble method, which I am interested in replicating within tidymodels framework.

[topepo]

This is a good idea.

I have a different thought about how to ensemble too that I won't go into now, but that led me to a few conclusions:

• New ensemble methods probably shouldn't go into stacks
• The stacks() |> add_candidates() sequence and fit_members() are great generic ways to process ensembles. blend_predictions() is also conceptually very good but really specific to stacking.

So I think that we should use as many stacks as possible for everything but creating new ensembles (in terms of who the members are).

The model_stack is very dialed in to using generalized linear models so we might want to define an alternate object type that isn't so specific.

Also, for some ensembles, we would probably use a simple average of the members' individual contributions, which is an "equation-free" approach. For this reason, we could avoid resampling the ensemble during the equivalent of blend_predictions() and instead use a submodel parameter for members. So, for Caruana, there would be predict(ens_obj, new_data, members = 1:10) that would allow us to externally validate the tuning parameters. Basically, the ensemble acts like any other parsnip model.

I said we "could avoid" it, but that doesn't mean that we shouldn't follow the same process as stacks currently does and lock the members in before fit_members().

I would suggest putting together some quick prototypes (with a lot of stacks:::internal_function() calls to see how feasible/invasive this would be for stacks.

Let me know if you think that this is a good idea, and we can combine some efforts.

[frankiethull]

I have been back and forth on how to handle a blend_caruana concept. Or if it would even make sense to have blend engines, such as blend_simple (average), blend_caruana, and so on.. I like your idea of treating each ensemble as it's own parsnip model though. I hadn't thought through the implementation until your comment but my request may be superseding stacks somewhat given the model_stack & stacking specifics.

But I did fork stacks & can test out some different ideas for caruana in particular & see if it makes sense or not for a PR. Let me know if you are working on anything in a branch & can share ideas/efforts.