Thank you everyone for the support. I have received nearly 40 signups and 1 paid user which is massive for me as I am still on the early stages of validating the product. So thank you everyone.
Next steps: Even though the signups are coming in I am tracking the usage of the app and I dont see many users running the evaluations and I need help. How should I get you to try and test the product.
My current thoughts are:
- User using their API keys means friction and would go away from the platform as can't test it immediately so perhaps allow free trial with my API key which involves unlimited runs with 3 test cases.
- Create a onboarding guide? Like the ones of enterprise softwares that says "Click here" "Next Steps": What tool can I use for this?
For both the options above will still need to inform them about the updates and hope they will signin again.
- Reach out to all 40 users for a 1-1 15 mins call and show them the product. Assuming 30% respond that 12 calls booked.
Do you have any suggestions? Keen on feedback. This is critical as I need to solve these issues before building next features i.e (Adding more models and multi model runs).
PromptPerf
Thank you Ajay for the review. The next phase is to add multiple model support so prompts can be tested against these and compared. Following this would be the ability to auto run the prompts against multiple models and temperatures with 3,5,10 runs to ensure the same prompt at the same temp on the same model provides consistent results (giving you the accuracy/consistent score on your prompt)