I proposed creating a smaller AI program with limited functionality and training it using a larger, more generic AI model. I had this epiphany a while earlier and had not pursued it yet. @gokayfem characterized the idea as an LLM extracting a mini expert in an MoE setup for the tasks currently being requested.
I said, “Had this epiphany a while ago, haven’t pursued it yet.”
@gokayfem characterized the idea this way: “In other words the llm extracts in MoE stole a mini expert only for the tasks it is now being asked for.”