Skip to content

Commit 11d2f43

Browse files
committed
Updates
1 parent fe979dd commit 11d2f43

File tree

2 files changed

+145
-9
lines changed

2 files changed

+145
-9
lines changed

โ€Žassets/logo.svgโ€Ž

Lines changed: 1 addition & 1 deletion
Loading

โ€Žslides.mdโ€Ž

Lines changed: 144 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -606,7 +606,68 @@ transition: slide-up
606606

607607
# โš’๏ธ How to Make Examples
608608

609-
## TBD
609+
<br>
610+
611+
### 1. Use real historical data
612+
613+
<br>
614+
615+
### 2. Reverse engineer ๐Ÿ”ฅ
616+
617+
<br>
618+
619+
```markdown
620+
Our context: <COMPANY_CONTEXT> + <USE_CASE_GOAL>
621+
622+
A) Generate a list of personas
623+
624+
B) Generate a list of topics
625+
626+
C) Given a <PERSONA> and a <TOPIC>,
627+
and a few real <EXAMPLES>,
628+
generate a list of typical user requests
629+
```
630+
631+
* Top tier models like **o3**, **o1 Pro** do this easily
632+
* Pro tip: <span style="color: darkgreen;">"Ask me the most important questions to help you do this task"</span>
633+
634+
---
635+
transition: slide-up
636+
---
637+
638+
# โžก๏ธ Choosing Examples
639+
640+
<br>
641+
642+
### ๐Ÿšซ Too many examples
643+
644+
<div style="padding-top: 0.5em">
645+
646+
- More noise, requires more thinking = more mistakes
647+
648+
</div>
649+
650+
<br>
651+
652+
### ๐Ÿšซ Too few examples
653+
654+
<div style="padding-top: 0.5em">
655+
656+
- Not always relevant = worse results
657+
658+
</div>
659+
660+
<br>
661+
662+
### โœ… Provide top 5-7 examples
663+
664+
<div style="padding-top: 0.5em">
665+
666+
- Generate a pool of examples (100-200)
667+
- Use RAG-like approach to choose the most relevant ones
668+
- Model is smart enough to extrapolate from these
669+
670+
</div>
610671

611672
---
612673
transition: slide-up
@@ -770,7 +831,7 @@ the same results for the given user question and DB schema?
770831
- Generate mock database that matches the production schema
771832
- Fill it with realistic synthetic data <span style="color: gray;">(or anonymize production data)</span>
772833
- Run Text-to-SQL on the user question, get new SQL query
773-
- Run both expected SQL and generated SQL at the mock DB
834+
- Run both `expected SQL` and `generated SQL` at the mock DB
774835
- Compare results โœ…
775836

776837
---
@@ -791,15 +852,90 @@ https://github.com/multinear/multinear
791852

792853
---
793854

855+
# ๐Ÿ›ก๏ธ Guardrails
856+
857+
Peace of mind
858+
859+
<div grid="~ cols-2 gap-4" style="font-size: 0.8em; line-height: 1.4;">
860+
861+
<div>
862+
863+
<div style="padding-bottom: 1.2em;">
864+
865+
**๐ŸŽฏ Accuracy & Reliability**
866+
* Fact-check, explain reasoning, user feedback
867+
868+
</div>
869+
870+
<div style="padding-bottom: 1.2em;">
871+
872+
**๐Ÿ”’ Security**
873+
* Access controls, validate input, monitor threats
874+
875+
</div>
876+
877+
<div style="padding-bottom: 1.2em;">
878+
879+
**๐Ÿ‘ค Privacy**
880+
* Anonymize PII, comply with GDPR & HIPAA
881+
882+
</div>
883+
884+
<div style="padding-bottom: 1.2em;">
885+
886+
**๐Ÿ—บ๏ธ Scope Definition**
887+
* Restrict off-topic requests, define usage policies
888+
889+
</div>
890+
891+
</div>
892+
893+
<div>
894+
895+
<div style="padding-bottom: 1.2em;">
896+
897+
**โš–๏ธ Ethical**
898+
* Moderate content, detect bias, align with ethics
899+
900+
</div>
901+
902+
<div style="padding-bottom: 1.2em;">
903+
904+
**โš™๏ธ Technical**
905+
* Handle errors, scale performance, optimize latency
906+
907+
</div>
908+
909+
<div style="padding-bottom: 1.2em;">
910+
911+
**๐Ÿ“ˆ Operational**
912+
* Monitoring, code versioning, human review
913+
914+
</div>
915+
916+
<div style="padding-bottom: 1.2em;">
917+
918+
**๐Ÿ›๏ธ Regulatory**
919+
* Adhere to regulations, protect IP, ensure accountability
920+
921+
</div>
922+
923+
</div>
924+
925+
</div>
926+
927+
---
928+
794929
# Takeaways
795930

796931
<br>
797932

798933
1. โญ Pinpoint *your* success criteria first
799934
2. ๐Ÿค” Make LLM think less, not more
800935
3. ๐Ÿ”Ž Reverse engineer examples and evals
801-
4. ๐Ÿ“ˆ Experiment, iterate, benchmark
802-
5. ๐Ÿš€ Production with confidence
936+
4. โœ… Use RAG to choose the most relevant examples
937+
5. ๐Ÿ“ˆ Experiment, iterate, benchmark
938+
6. ๐Ÿš€ Production with confidence
803939

804940
<style>
805941
div {
@@ -817,7 +953,7 @@ https://github.com/multinear/multinear
817953

818954
<br>
819955

820-
- [Multinear Site](https://multinear.com)
956+
- [Multinear Site](https://multinear.com) <img src="./assets/logo.svg" style="width: 1.5em; height: 1.5em; display: inline-block; margin-top: -0.2em; margin-left: 0.2em;"></img>
821957
- [Multinear Platform](https://github.com/multinear/multinear)
822958
- [Uber Text-to-SQL](https://www.uber.com/en-GB/blog/query-gpt/)
823959
- [LinkedIn Text-to-SQL](https://www.linkedin.com/blog/engineering/ai/practical-text-to-sql-for-data-analytics)
@@ -845,15 +981,15 @@ background: ./assets/thanks.jpg
845981

846982
<br>
847983

848-
๐Ÿ”ธ Register for deep-dive workshop
984+
๐Ÿ”ธ Register for individual workshop
849985

850986
๐Ÿ”ธ Follow us on LinkedIn and X
851987

852-
๐Ÿ”ธ Subscribe to the newsletter
988+
๐Ÿ”ธ Subscribe to our newsletter
853989

854990
<br>
855991

856-
๐Ÿ”ธ Use Multinear
992+
๐Ÿ”ธ Use Multinear <img src="./assets/logo.svg" style="width: 1.5em; height: 1.5em; display: inline-block; margin-top: -0.2em; margin-left: 0.2em;"></img>
857993

858994
<img src="./assets/multinear.png" style="width: 20em; display: block; margin-left: auto; margin-right: auto;"></img>
859995

0 commit comments

Comments
ย (0)