Code Switching

Emergent Behavior

In this episode of Emergent Behavior, @8teapi talks with Justin Junyang Lin, Chief Evangelist Officer of Alibaba Qwen Project. Joined by guest host Eugene Cheah, CEO of Recursal.AI, they talk about how Alibaba's Qwen 2 tackles multilingual challenges, including code-switching and the unique complexities of Chinese data.

🔥 Apply to join over 400 founders and Execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj

Explore the impact of open-source LLMs like Alibaba's Qwen 2, and how it's driving innovation in AI development.

--

RECOMMENDED PODCASTS:

🎙️ This Won't Last - Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel. They unpack their hottest takes on the future of tech, business, venture, investing, and politics.

Apple Podcasts: https://podcasts.apple.com/us/podcast/id1765665937

Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz

YouTube: https://www.youtube.com/@ThisWontLastpodcast

–

FOLLOW ON X:

@8teAPi (Ate)

@JustinLin610 (Junyang)

@picocreator (Eugene)

@TurpentineMedia

--

LINKS:

Alibaba Qwen Project:

https://www.alibabacloud.com/en/solutions/generative-ai/qwen?_p_lc=1

--

TIMESTAMPS:

(00:00) Introduction

(04:36) Qwen's Development Journey

(08:00) Data Curation & Coding Capabilities

(11:00) The Role of Evaluation

(14:00) Evolution of Pre-training and Evaluation

(17:00) Open Source vs. Commercial Groups

(22:00) Data Contamination

(24:00) Model Sizing and Computational Constraints

(28:00) Multi-lingual Capabilities

(31:00) Tokenizers and Language-Specific Considerations

(34:00) Code Switching and Data Filtering

(38:00) Code Switching, Dialects, and Model Size

(42:00) User Feedback and Model Development

(46:00) Challenges with Chinese Datasets

(52:00) Language Variation and Team Development

(58:00) Hiring and Team Dynamics

(1:03:00) Diversity and Production Considerations

(1:07:00) Production Impact and Collaboration

(1:13:00) Wrap