@@ -606,7 +606,68 @@ transition: slide-up
606606
607607# โ๏ธ How to Make Examples
608608
609- ## TBD
609+ <br >
610+
611+ ### 1. Use real historical data
612+
613+ <br >
614+
615+ ### 2. Reverse engineer ๐ฅ
616+
617+ <br >
618+
619+ ``` markdown
620+ Our context: <COMPANY_CONTEXT> + <USE_CASE_GOAL>
621+
622+ A) Generate a list of personas
623+
624+ B) Generate a list of topics
625+
626+ C) Given a <PERSONA > and a <TOPIC >,
627+ and a few real <EXAMPLES >,
628+ generate a list of typical user requests
629+ ```
630+
631+ * Top tier models like ** o3** , ** o1 Pro** do this easily
632+ * Pro tip: <span style =" color : darkgreen ;" >"Ask me the most important questions to help you do this task"</span >
633+
634+ ---
635+ transition: slide-up
636+ ---
637+
638+ # โก๏ธ Choosing Examples
639+
640+ <br >
641+
642+ ### ๐ซ Too many examples
643+
644+ <div style =" padding-top : 0.5em " >
645+
646+ - More noise, requires more thinking = more mistakes
647+
648+ </div >
649+
650+ <br >
651+
652+ ### ๐ซ Too few examples
653+
654+ <div style =" padding-top : 0.5em " >
655+
656+ - Not always relevant = worse results
657+
658+ </div >
659+
660+ <br >
661+
662+ ### โ
Provide top 5-7 examples
663+
664+ <div style =" padding-top : 0.5em " >
665+
666+ - Generate a pool of examples (100-200)
667+ - Use RAG-like approach to choose the most relevant ones
668+ - Model is smart enough to extrapolate from these
669+
670+ </div >
610671
611672---
612673transition: slide-up
@@ -770,7 +831,7 @@ the same results for the given user question and DB schema?
770831- Generate mock database that matches the production schema
771832- Fill it with realistic synthetic data <span style =" color : gray ;" >(or anonymize production data)</span >
772833- Run Text-to-SQL on the user question, get new SQL query
773- - Run both expected SQL and generated SQL at the mock DB
834+ - Run both ` expected SQL ` and ` generated SQL ` at the mock DB
774835- Compare results โ
775836
776837---
@@ -791,15 +852,90 @@ https://github.com/multinear/multinear
791852
792853---
793854
855+ # ๐ก๏ธ Guardrails
856+
857+ Peace of mind
858+
859+ <div grid =" ~ cols-2 gap-4 " style =" font-size : 0.8em ; line-height : 1.4 ;" >
860+
861+ <div >
862+
863+ <div style =" padding-bottom : 1.2em ;" >
864+
865+ ** ๐ฏ Accuracy & Reliability**
866+ * Fact-check, explain reasoning, user feedback
867+
868+ </div >
869+
870+ <div style =" padding-bottom : 1.2em ;" >
871+
872+ ** ๐ Security**
873+ * Access controls, validate input, monitor threats
874+
875+ </div >
876+
877+ <div style =" padding-bottom : 1.2em ;" >
878+
879+ ** ๐ค Privacy**
880+ * Anonymize PII, comply with GDPR & HIPAA
881+
882+ </div >
883+
884+ <div style =" padding-bottom : 1.2em ;" >
885+
886+ ** ๐บ๏ธ Scope Definition**
887+ * Restrict off-topic requests, define usage policies
888+
889+ </div >
890+
891+ </div >
892+
893+ <div >
894+
895+ <div style =" padding-bottom : 1.2em ;" >
896+
897+ ** โ๏ธ Ethical**
898+ * Moderate content, detect bias, align with ethics
899+
900+ </div >
901+
902+ <div style =" padding-bottom : 1.2em ;" >
903+
904+ ** โ๏ธ Technical**
905+ * Handle errors, scale performance, optimize latency
906+
907+ </div >
908+
909+ <div style =" padding-bottom : 1.2em ;" >
910+
911+ ** ๐ Operational**
912+ * Monitoring, code versioning, human review
913+
914+ </div >
915+
916+ <div style =" padding-bottom : 1.2em ;" >
917+
918+ ** ๐๏ธ Regulatory**
919+ * Adhere to regulations, protect IP, ensure accountability
920+
921+ </div >
922+
923+ </div >
924+
925+ </div >
926+
927+ ---
928+
794929# Takeaways
795930
796931<br >
797932
7989331 . โญ Pinpoint * your* success criteria first
7999342 . ๐ค Make LLM think less, not more
8009353 . ๐ Reverse engineer examples and evals
801- 4 . ๐ Experiment, iterate, benchmark
802- 5 . ๐ Production with confidence
936+ 4 . โ
Use RAG to choose the most relevant examples
937+ 5 . ๐ Experiment, iterate, benchmark
938+ 6 . ๐ Production with confidence
803939
804940<style >
805941 div {
@@ -817,7 +953,7 @@ https://github.com/multinear/multinear
817953
818954<br >
819955
820- - [ Multinear Site] ( https://multinear.com )
956+ - [ Multinear Site] ( https://multinear.com ) < img src = " ./assets/logo.svg " style = " width : 1.5 em ; height : 1.5 em ; display : inline-block ; margin-top : -0.2 em ; margin-left : 0.2 em ; " ></ img >
821957- [ Multinear Platform] ( https://github.com/multinear/multinear )
822958- [ Uber Text-to-SQL] ( https://www.uber.com/en-GB/blog/query-gpt/ )
823959- [ LinkedIn Text-to-SQL] ( https://www.linkedin.com/blog/engineering/ai/practical-text-to-sql-for-data-analytics )
@@ -845,15 +981,15 @@ background: ./assets/thanks.jpg
845981
846982<br >
847983
848- ๐ธ Register for deep-dive workshop
984+ ๐ธ Register for individual workshop
849985
850986๐ธ Follow us on LinkedIn and X
851987
852- ๐ธ Subscribe to the newsletter
988+ ๐ธ Subscribe to our newsletter
853989
854990<br >
855991
856- ๐ธ Use Multinear
992+ ๐ธ Use Multinear < img src = " ./assets/logo.svg " style = " width : 1.5 em ; height : 1.5 em ; display : inline-block ; margin-top : -0.2 em ; margin-left : 0.2 em ; " ></ img >
857993
858994<img src =" ./assets/multinear.png " style =" width : 20em ; display : block ; margin-left : auto ; margin-right : auto ;" ></img >
859995
0 commit comments