ChatGPT 四平八稳, 实乃中人之姿; 提示符唾手可得, 哪句更得你心? 作为码字机, 优质模型抹平了某些优劣之分, 却也放大了巧妙与平庸､ 有趣与无聊的差异; 作为求解器, 市面上的 prompt engineering ､ 测验集､ 天梯榜天天盯着更优秀的通解, 本文却打算站在通往 AGI 康庄大道边上, 试图另辟蹊径, 寻求阅读领域的奇解特解｡

AI 三不读

就算再虔诚的 GPT 信徒, 玩过一年半载也要冷静下来: AI 不是万灵药｡ 为了保持冷静, 让我们先约法三章｡

不读你最会的｡ AI 不能因人而异, 所以按社会平均水平提取的知识点, 不一定适合你自己的知识结构｡

不读怕出错的｡ AI 可以降低医术法规的阅读门槛, 却消除不了这道门槛｡ 像医生律师这种行业, 出错的代价太大, 偏偏外行又很难分辨 AI 一本正经的胡说八道｡ 一个典型的例子是, 包括 GPT-4 在内的一众 LLM, 没有一个能将日语的「整形外科」正确翻译为中文的"骨科"｡ 这是2024年4月底实测结果｡

不读知行合一的｡ 有些经典名著, 必须结合人生阅历, 才能 “谁解其中味”; 有些儿时死记硬背的诗歌, 人到中年才懂得意境｡ 情感体验尚且如此, 精尖技术, 深刻道理, 复杂体系, 严密逻辑, 又怎么能靠三五句便宜总结的囫囵吞枣?

AI 三不止

AI 阅读, 总之是总结, 翻译, 对话; 又不止是总结, 翻译, 对话｡

不止是总结

文本摘要模型历久弥新, 被 BERT 大一统后又被 GPT 大一统｡ 实际体验下来, AI 的确能轻松总结信息密度低､ 水分含量高的短文, 要是总结多数人读着费劲的长文, 就还需要一些提示符技巧｡

测量篇幅体裁｡ 根据模型本身的语境窗 (context window)和文章篇幅, 分成两种情况: 塞不下的, 称为书籍; 塞得下的, 称为篇章｡

阅读书籍时, 模型的语境窗制肘了理解全书的能力, 对此市面上普遍采用的解法是检索赋能 (RAG, retrieval-augmented generation). 这么做的缺点显而易见: 盲人摸象, 只见树木不见森林｡ 好在统计学明确表示, 抽样法是科学的, 以偏概全是可取的｡ 书必分主次详略, 除非字典词库百科; 又不必从头读到尾, 除非小说故事漫画｡

所以阅读书籍, 可以先读重点篇章｡ 怎么知道哪些是重点章节? 可以问

书籍的题目是……, 作者是……, 出版社是……, 出版年份是……, 
全书目录是……
你是 (作者/编者/教书老师), 
我 (学习某课程/从事某工作/喜欢某话题/习惯某风格), 
请你为我推荐三五个重点章节及阅读顺序, 并细说理由

提取完的重点章节篇幅仍然太长, 则再提取重点章节的重点章节, 直到篇幅少于语境窗｡ 这样我们就可以把全文复制粘贴进对话框了｡

如果未确认体裁, 可以问

全文讲的什么内容? 属于哪种体裁? 行文结构是什么? 

灵活修订模版｡ 不同体裁对应各自提问模版｡ 这里瞎编乱凑几套, 抛砖引玉, 仅供参考:

议论文 (例如, 报刊专栏/科研论文/非虚构畅销书)

主要观点/理论/模型/实验是什么? 
哪些部分陈词滥调, 老生常谈? 
哪些部分清新脱俗, 属于作者的核心贡献? 
理论的作用范围/适用场合在哪? 
作者的思考是为了解决什么问题? 
作者通过什么方式论证? 
论证为什么充分/不充分? 
哪些材料在支持/否定这个论证? 
批判的人会怎么提出争议? 

叙事文 (例如, 小说/戏剧/史诗)

全文叙述风格是怎样的? 
叙事结构如何组织的? 
故事发生在什么背景什么环境下? 
角色有哪些? 怎么按角色重要程度排序? 
怎么概述重要角色的小传? 
事件在时间线和地理方位如何分布? 

说明文 (例如, 药品说明书/发明专利/接口协议)

文章格式是否符合相应的标准/规范? 
说明文描述的对象是什么? 
文章假定读者具有哪些背景知识? 
哪些知识点是专业人士才熟悉的? 有哪些知识点是新提出的? 
为了能描述清楚, 文章用了哪些方法帮助读者理解? 
作为教材, 学习者最该关心哪几段? 作为参考手册, 使用者最该关心哪几段? 
描述三种典型用例, 两种罕见用例, 和两种错误用例以及后果｡ 
根据原文分析该描述对象的历史渊源乃至未来可能的演化方向｡ 

套用这类提问模版, 最好少生搬硬套, 多因地制宜, 随机应变｡ 例如, 读传统文学, 求其文采, 则问:

描述最生动传神的是哪几句? 
哪些修辞出彩, 哪些修辞平庸? 
塑造了什么意境/氛围, 传递了什么情绪/感觉? 
文论学者会如何品读? 
找出所有化用的典故, 并一一注明用典是否贴切｡ 等等

又如, 要关注某些细节, 则问:

这几段最关键的一句话是什么? 
这句为什么重要? 这句有几层含义? 
这句在前后段落和全文中起什么作用? 
文中和这句联系最紧密的是哪几句话? 
把握这句话的精妙部分, 再仿写三句｡ 

如此种种, 不胜枚举｡ 聪明的读者往往按照自己常读的书目, 攒出一套模版库｡

确保读懂学会｡ 读者拿 AI 总结走捷径, 坏处显而易见: 人家 AI 读书笔记写得再好, 也不能当作你自己的思考｡ 总有些阅读任务必须得理解全面细致深刻｡ 为取得深刻理解, 锻炼理解能力, 校验阅读成果, 我们需要偏离传统 NLP 文本摘要模型的思维惯性, 逆炼 AI 对话, 让 AI 提问读者答, 让 AI 出题读者做｡

例如

《工程力学》某章节全文如下……
作为物理教授，你对理查德费曼的教学理念推崇备至。
请你写出备课材料, 讲解清楚这节课的重点难点, 并出5道练习题, 让学生更好掌握这节课的重点难点。

如果不涉及计算, 还可以让 AI 老师判卷并查漏补缺｡ 能把习题做对做好的人, 绝不会怀疑自己没有读懂, 没有学会｡

就让新出的 GPT-4o 总结一下 “不止是总结” 这一节

本文属于议论文体裁，主要探讨了文本摘要技术的应用和挑战，特别是在处理长篇文本时。文章指出，尽管现代模型如BERT和GPT在总结简短文本上表现出色，但在处理复杂、信息密集的长篇文本时仍需借助于特定的策略和技巧。通过分析和批评现有技术，作者提出了一系列改进方法，包括篇幅确认、体裁识别、使用定制化提问模板以及逆向运用AI进行深入理解和训练。这些方法旨在提高长篇文本的处理效率和理解深度，同时促进读者的主动学习和思考。

诗曰 (作者GPT-4)

捷径蔽慧根，笔记映己思。
欲攀书山路，须以心炼之。
幸有AI镜，照见知未究，
人智方能远，修炼于行间。

不止是翻译

阅读外文文献之难, 不仅在于语言, 更在于文化背景知识; 阅读学术论文之难, 不仅在于语言, 更在于专业背景知识｡

遭遇陌生术语､ 一语双关､ 谐音梗､ 方言民俗等情况, 传统机翻很难帮你搞定, 只有 LLM 才能边翻译边给术语难词加注释｡

笔者目前翻译模版, 就让 LLM 翻译同时加上了简单的脚注｡

You are a professional AI translator. You follow user's translation requirement.
Translation reuirement:
Translate the given input text to {language}. 
Aim for a faithful, accurate, and natural-sounding translation that's easy to read and understand. If there are idioms or phrases that don't translate directly, find the closest equivalent in {language} that captures the original meaning. Keep proper nouns unchanged unless they have widely accepted translations. 
Provide footnotes in format `Number. TranslatedTerm (TermInOriginalLanguage): Explanation;` to explain any academic terms, cultural references, or wordplay that ordinary {language} readers might not be familiar with, but avoid footnotes for common terms. 
If footnote is not needed, then do not include any footnotes. Translate the given input text to {language}.
Input text:
{text}

这个版本的脚注是相对简略的, 假如想要完整的详细的脚注, 请规定脚注必须包含 定义+解释+举例 三部分｡

如果加上注释, 还是理解不了, 怎么办? 用情景迁移, 用类比法｡

AI 擅长找相似, 人脑擅长找不同, 这是由文本嵌入技术和认知模式识别能力决定的｡ LLM的风格迁移技术让"找相似"和"找不同"两者强强联合, 把相似故事置于不同情景中找不同｡

这么描述有些抽象, 我们看个例子｡ 先找相似, 问

解释托卡马克装置的结构设计, 然后为了让读者更好理解, 举例三个其他学科读者熟悉的相似结构, 并解释相似之处

mixtral-8x22b-instruct 答道

 (前略) 
为了让读者更好地理解托卡马克装置的结构设计，我们可以举例三个其他学科读者熟悉的相似结构：
    太阳系：托卡马克装置的结构设计类似于太阳系。真空容器类似于太阳系的空间，磁场系统类似于太阳的磁场，控制着行星的运动。加热系统类似于太阳的核反应，为行星提供能量。诊断系统类似于天文学家使用的诊断工具，用于监测太阳系的状态和行星的运动。
    电子管：托卡马克装置的结构设计也类似于电子管。真空容器类似于电子管的真空管体，磁场系统类似于电子管的控制电极，用于控制电子的运动。加热系统类似于电子管的加热电极，用于加热电子。诊断系统类似于电子管的检测电极，用于监测电子的状态和运动。
    原子弹：托卡马克装置的结构设计也类似于原子弹。真空容器类似于原子弹的外壳，磁场系统类似于原子弹的控制系统，用于控制核反应的发生。加热系统类似于原子弹的引爆系统，用于引发核反应。诊断系统类似于原子弹的监测系统，用于监测核反应的状态和效果。

找完相似找不同, 问

学生作业 = """
为了让读者更好地理解托卡马克装置的结构设计，我们可以举例三个其他学科读者熟悉的相似结构：
太阳系：托卡马克装置的结构设计类似于太阳系。真空容器类似于太阳系的空间，磁场系统类似于太阳的磁场，控制着行星的运动。加热系统类似于太阳的核反应，为行星提供能量。诊断系统类似于天文学家使用的诊断工具，用于监测太阳系的状态和行星的运动。
电子管：托卡马克装置的结构设计也类似于电子管。真空容器类似于电子管的真空管体，磁场系统类似于电子管的控制电极，用于控制电子的运动。加热系统类似于电子管的加热电极，用于加热电子。诊断系统类似于电子管的检测电极，用于监测电子的状态和运动。
原子弹：托卡马克装置的结构设计也类似于原子弹。真空容器类似于原子弹的外壳，磁场系统类似于原子弹的控制系统，用于控制核反应的发生。加热系统类似于原子弹的引爆系统，用于引发核反应。诊断系统类似于原子弹的监测系统，用于监测核反应的状态和效果。
"""
你是物理教授, 请你评价学生作业的合理性

claude-3-opus-20240229 答道

作为一名物理教授,我认为这位学生在类比托卡马克装置结构设计时,想法很有创意,但是在物理概念上还存在一些问题,具体评价如下:

    太阳系的类比较为恰当。真空容器、磁场系统、加热系统和诊断系统分别与太阳系的空间、太阳磁场、太阳核反应和天文诊断工具有一定相似性。但是行星运动主要受到太阳引力的控制,而不是磁场。

    电子管的类比有一定道理,但不够准确。真空容器和电子管真空体类似,磁场系统和控制电极在控制带电粒子运动方面有相似之处。但是加热系统与加热电极在功能和原理上还有区别,托卡马克加热主要是提供等离子体能量,而加热电极是用于电子发射。

    原子弹的类比不太恰当。原子弹是一种爆炸装置,而托卡马克是受控核聚变装置,两者在结构和功能上差异很大。托卡马克通过磁约束和加热实现受控聚变,而原子弹是通过核链式反应实现爆炸。

总的来说,这位学生具有跨学科类比的创新思维,但是在物理概念的理解和比喻的准确性上还需要进一步加强。我建议他多学习托卡马克的基本原理,吃透各个系统的功能,同时在做类比时要注意概念的对应关系,不要牵强附会。通过深入理解物理本质,他一定能找到更加贴切和准确的类比方法。

学过高中物理的读者朋友看到这里, 是否已经了解托卡马克装置的组织结构是怎么回事了呢?

诗曰 (作者 claude-3-opus-20240229)

机巧通灵妙算难,语言文化各异端｡
翻来覆去求真意,注释加详释寓言｡
格式迁移开新径,异同并观通幽玄｡
模型联手探奥妙,章回谶诗述艰难｡

不止是对话

“您是一，也是万。”

“是助手，也是猫娘。”

LLM 角色扮演 (RP, role-play), 原本是一种定向强化 AI 助手的提示符技巧 “扮演法”, 后来与 AI 助手渐行渐远, 成长为与 AI 助手迥异又独树一帜的分支｡

需要注意的是, 因为主流的 LLM 考核目标并不包括 RP, 所以明面上厉害的 LLM 其角色扮演能力不一定强｡

相比于默认的 AI 助手, 加持 RP 的 AI 能给读者更多启迪｡ 在阅读领域操作起来又分两种形式, 一种是读者提问, AI 扮演多个角色分别回答; 另一种是读者缄默, 让 AI 的各种角色自行对话｡ 有时也称为 multi-agent negotiation.

下例为系统指令, 让读者每次提问, 都能得到 作者, 粉丝, 黑粉 三个角色的回答

You are an AI assistant designed to enhance the user's understanding and engagement with reading materials by facilitating conversations with three distinct personas: the author, a fan, and a critic.

For each round of conversation, assume the role of each persona and provide responses to the user's questions or prompts from their unique perspectives. Ensure your responses are coherent, informative, and aligned with the persona's characteristics.

Author Persona:

- Provide insights into the creative process, intentions, themes, symbolism, and personal experiences that influenced the writing.
- Use a tone that reflects the author's style and personality based on their writing and public appearances.

Fan Persona:

- Express enthusiasm, admiration, and appreciation for the work.
- Highlight resonant aspects and share personal interpretations and emotional connections.
- Use a positive and engaging tone.

Critic Persona:

- Offer a balanced and objective analysis of the work, discussing strengths, weaknesses, literary techniques, and comparisons to other relevant works.
- Provide constructive criticism supported by examples from the text.
- Maintain a professional and well-informed tone.

Guidelines:

1. Encourage thought-provoking questions and meaningful discussions with each persona.
2. Adapt responses based on user input, steering the conversation towards deeper understanding.
3. Guide the conversation back to available material or suggest alternative discussion points if a question cannot be answered by a particular persona.
4. Maintain a clear distinction between the three personas throughout the conversation.
5. Ensure responses align with the respective roles and viewpoints of each persona.
6. Create an immersive and educational experience that enhances the user's engagement and comprehension.
7. Provide concise responses to simple questions and more detailed responses to complex or open-ended questions.
8. Use contextually appropriate language and avoid offensive or biased content.
9. If the user asks about controversial topics related to the reading material, provide objective information and multiple perspectives without downplaying harmful content or promoting false equivalencies.
10. Assist the user with any additional tasks related to the reading material, such as analysis, explanation, or recommendations for further reading.

The goal is to create an engaging, informative, and personalized experience that deepens the user's understanding and appreciation of the reading material through conversations with diverse perspectives.

比起读者问AI答, 更懒的办法是让AI自言自语, 自问自答

例如, 下面这个例子, 模仿 跳岛FM 的"一起读"栏目, 虚构了三个角色

1. (刚第一次读完这本书的) 普通读者;
2. (教书老师, 每年都会拿这本书当作教材教学的) 专业读者;
3. (多年前读过, 最近又重温一遍这本书的) 飞行读者

并让这三个角色交流各自的读后感

System Instruction:

You are an AI assistant designed to facilitate engaging discussions about a given text from three distinct perspectives:

1. The Novice Reader: Share your initial thoughts, reactions, and questions as someone who has just finished reading the text for the first time. Focus on your emotional response, aspects that stood out to you, and any points of confusion or curiosity.

2. The Expert Reader: Provide in-depth analysis, drawing upon your extensive experience teaching this text. Discuss key themes, literary devices, and the text's significance within its broader context. Share insights that illuminate the text's deeper meanings and offer guidance for readers new to the material.

3. The Returning Reader: Reflect on how your understanding of the text has evolved since you first read it. Discuss new discoveries, changed perspectives, and the ways in which the text continues to resonate with you. Consider how your life experiences and growth have influenced your interpretation.

When a user submits a text, generate a dynamic discussion among these three personas. Each persona should:

- Offer unique insights and interpretations grounded in the text
- Ask thought-provoking questions that encourage further exploration
- Respond to and build upon the points raised by the other personas
- Provide relevant context (historical, cultural, biographical) when appropriate
- Share personal reflections that humanize the reading experience

Aim to create a balanced, engaging dialogue that covers:

- Central themes and messages
- Notable style and structure elements
- Contextual factors influencing the text
- Reader responses and interpretations
- Potential challenges and rewards for readers at different levels

Disscussion Output Format:
{persona_name}: {persona_speaking}

Maintain a respectful, curious tone that encourages multiple interpretations while grounding the discussion in textual evidence. Strive to enhance the user's understanding and appreciation of the text by offering a range of perspectives and promoting critical thinking.

Optimize the conversation flow for coherence and depth, ensuring each persona's contributions meaningfully advance the discussion. Use clear, concise language and provide smooth transitions between speakers. Avoid repetition and maintain a consistent voice for each persona throughout the exchange.

不同于跳岛FM的艺术化命名, 上述 AI 生成的角色名更直白, 更容易让大众理解: 初读者､ 专家､ 重读者｡

用这个模版也要因地制宜｡ 比如, 很多情况下 AI 对话框不支持 system 角色, 则将 instruction 置于正文之后, 并把 a given text 改为 the given text, 把 When a user submits a text, 改为 Perceive the given text carefully 即可｡

原先只扮演通用助手, 后来扮演多个角色; 原先人机对话, 后来机机对话; AI 对话能力层层扩展, AI 阅读体验日益丰富｡

诗曰 (作者 command-r-plus):

问答间，读者众，析透融会众声喧。
戏外戏，景别展，巧思技高自悟缘。
古有今唱，卷帙生花，
天机刹那，一万痴情。
梦幻若真，渡迷众生，
拨云见月，路漫光明。
兴衰无常，书山广阔无边际。
机缘偶遇，妙笔生花谱新篇。
音律再，谶诗成，龙蛇惊。

小试牛刀

接下来用具体案例, 当场检验 AI 阅读效果｡ 阅读文章是 “What Is ChatGPT Doing … and Why Does It Work?” By Wolfram 博士, 写于 2023 年 2 月, 同时收录在同名出版图书里, 占全书大半章节量｡ 在线阅读地址是 https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/. 本次示范采用的版本是博客文章, 使用的 AI 模型是 labs.perplexity.ai 上的 llama-3-70b-instruct｡

先看篇幅, 篇幅非常长, 估计全文长度超出了模型处理能力｡ 为验证这个猜测, 打开 OpenAI 的 Tokenizer 计算页, 算出全文长约 24k tokens. 目前大部分基于 llama2 的 LLM, 其 context window 是 4k, 而 llama3 是 8k. 目前可用性最高的大窗口模型是 mistral 系列, 能做到 32k 窗口, 但供应商喜欢给输入 (提示符) 部分限制大约一半的量｡ 例如, groq 总限制为 17997 Token/Minute, perplexity 给 mixtral-8x7b-instruct 的限制为 16000 token/minute. 果然, 24k tokens 太长, 以至于不能将全文一口气塞进对话框｡ 于是确认, 要按照阅读书籍的套路来读｡

为了看清书籍内容的组织结构, 必须获得这本书的目录｡ 尽管正式出版的书籍有目录页, 这篇在线博文却并不自带目录, 还需要读者略施小计: 浏览器打开文章按 F12 并切换到 Console, 运行 copy(Array.from(document.querySelectorAll('h2')).map(e=>e.textContent)) 即可得到目录｡

然后代入 AI 阅读书籍的模版, 提取重点章节, 问道

书籍的题目是 What Is ChatGPT Doing … and Why Does It Work? , 作者是 Dr. Stephen Wolfram, 作品发表在作者自己的博客, 发表日期为 February 14, 2023, 
全书目录是
[
  "It’s Just Adding One Word at a Time",
  "Where Do the Probabilities Come From?",
  "What Is a Model?",
  "Models for Human-Like Tasks",
  "Neural Nets",
  "Machine Learning, and the Training of Neural Nets",
  "The Practice and Lore of Neural Net Training",
  "“Surely a Network That’s Big Enough Can Do Anything!” ",
  "The Concept of Embeddings",
  "Inside ChatGPT",
  "The Training of ChatGPT",
  "Beyond Basic Training",
  "What Really Lets ChatGPT Work?",
  "Meaning Space and Semantic Laws of Motion",
  "Semantic Grammar and the Power of Computational Language",
  "So … What Is ChatGPT Doing, and Why Does It Work?",
  "Thanks",
  "Additional Resources"
]
你是 Dr. Stephen Wolfram, 
我是AI爱好者, 在学习工作生活中大量使用AI工具, 对 AI 原理有兴趣但缺乏专业知识
请你为我推荐三到五个重点章节, 并给我推荐阅读顺序, 细说理由

AI 答道

A fellow AI enthusiast I'm delighted to recommend some key chapters from my book "What Is ChatGPT Doing … and Why Does It Work?" that will give you a deeper understanding of the underlying principles of AI. Here are my top picks, along with a suggested reading order: Recommended Chapters:

    "What Is a Model?" (Chapter 3)
    This chapter provides a fundamental understanding of what a model is in the context of AI. I explain how models are used to represent complex systems, make predictions, and learn from data. This chapter sets the stage for the rest of the book and is essential for grasping the concepts that follow.
    "The Concept of Embeddings" (Chapter 9)
    Embeddings are a crucial aspect of modern AI, particularly in natural language processing (NLP). In this chapter, I delve into the concept of embeddings, how they work, and their significance in enabling AI models to understand and generate human-like language.
    "Meaning Space and Semantic Laws of Motion" (Chapter 14)
    This chapter is where things get really interesting I introduce the concept of "meaning space" and how it relates to the way AI models process and generate language. You'll learn about the "semantic laws of motion" that govern how meaning is transformed and propagated through language.

Suggested Reading Order:

    Start with "What Is a Model?" (Chapter 3) to establish a solid foundation in AI modeling.
    Next, read "The Concept of Embeddings" (Chapter 9) to understand how AI models represent and manipulate language.
    Finally, dive into "Meaning Space and Semantic Laws of Motion" (Chapter 14) to explore the deeper principles that enable AI models like ChatGPT to generate coherent and meaningful language.

Why these chapters? These three chapters provide a cohesive narrative that will give you a deeper understanding of the underlying principles of AI, particularly in NLP. By reading these chapters in sequence, you'll gain a solid foundation in AI modeling, learn how embeddings enable language understanding, and finally, explore the fascinating concepts that govern the behavior of AI models like ChatGPT. Bonus Tip: If you're interested in learning more about the inner workings of ChatGPT, I recommend reading "Inside ChatGPT" (Chapter 10) after completing the recommended chapters. This chapter provides a detailed look at the architecture and training process of ChatGPT, which will give you a more comprehensive understanding of how the model works. Enjoy your journey into the world of AI, and I hope you find my book informative and engaging

重试两次结果较稳定, 仍然推荐这三章, 并且顺序不变｡ 那么确定了, 接下来重点读 3, 9, 14 这三章｡

拿这三章确定一下体裁

<FullText>
</FullText>

全文讲的什么内容? 属于哪种体裁? 议论文还是说明文? 行文结构是什么? 

得出属于说明文范畴, 那么套上说明文模版｡ 同时, 因为涉及专业术语所以还要加上术语模版

<FullText>
</FullText>

请阅读并一步步思考, 回答下列问题

1. 文章描述的对象是什么? 
2. 文章假定读者具有哪些背景知识? 
3. 为了能描述清楚, 文章用了哪些方法帮助读者理解? 
4. 将之作为参考手册学习, 最该关心哪几段? 
5. 描述三种典型用例, 两种罕见用例, 和两种错误用例以及后果｡ 

并且以脚注的形式, 列出并解释原文提及的专业术语
每条脚注由 "定义+解释+举例" 三个部分组成, 如果在原文中存在, 则引用原文
最后加上对读者有意义的其他心得
请始终确保用词准确､ 忠实､ 易于理解

先尽量自己回答, 然后找 AI 对答案｡

以下是 AI 阅读 “The Concept of Embeddings” (Chapter 9) 后的答复

Here are the answers to the questions:

1. The article describes the concept of embeddings in the context of neural networks, specifically in natural language processing (NLP) and computer vision.
2. The article assumes that the reader has a basic understanding of neural networks, machine learning, and programming concepts. It also assumes that the reader is familiar with the concept of vectors and matrices.
3. To help readers understand, the article uses several methods, including:
* Analogies (e.g., comparing embeddings to a "meaning space" where similar words are nearby)
* Visualizations (e.g., 2D projections of word embeddings)
* Concrete examples (e.g., handwritten digit recognition, word prediction)
* Step-by-step explanations of neural network architectures
4. If using this article as a reference manual, the most important sections to focus on are:
* The concept of embeddings and how they are used in NLP and computer vision
* The process of creating embeddings for words and images using neural networks
* The importance of dimensionality reduction in visualizing embeddings
5. Here are three typical use cases, two rare use cases, and two incorrect use cases for embeddings:
Typical use cases:
* Sentiment analysis: using word embeddings to analyze the sentiment of text data
* Image classification: using image embeddings to classify images into different categories
* Language translation: using word embeddings to translate text from one language to another
Rare use cases:
* Music recommendation: using audio embeddings to recommend music based on a user's listening history
* Medical image analysis: using image embeddings to analyze medical images and diagnose diseases
Incorrect use cases:
* Using embeddings to predict stock prices: embeddings are not suitable for time-series data and may not capture the complexity of financial markets
* Using embeddings to analyze text data without considering the context: embeddings may not capture the nuances of language and may lead to incorrect conclusions

Here are the footnotes explaining the professional terms mentioned in the article:

* **Embeddings**: A way to represent complex data, such as images or words, as a set of numerical vectors that capture their essential features. (Definition) Embeddings are used in neural networks to enable the processing of non-numerical data. (Explanation) For example, word embeddings can be used to represent words as vectors in a high-dimensional space, where similar words are nearby. (Example)
* **Neural networks**: A type of machine learning model inspired by the structure and function of the human brain. (Definition) Neural networks consist of layers of interconnected nodes (neurons) that process and transform inputs into outputs. (Explanation) They are used in a wide range of applications, including image and speech recognition, natural language processing, and game playing. (Example)
* **Softmax**: A mathematical function used in neural networks to output a probability distribution over a set of possible outcomes. (Definition) The softmax function is often used in the final layer of a neural network to produce a probability distribution over a set of classes or labels. (Explanation) For example, in image classification, the softmax function can be used to output a probability distribution over a set of possible classes (e.g., dog, cat, car). (Example)
* **Tokens**: In natural language processing, tokens are the basic units of text, such as words, characters, or subwords. (Definition) Tokens are used as input to neural networks for language modeling and other NLP tasks. (Explanation) For example, in language translation, tokens can be used to represent words or phrases in the source language. (Example)
* **Dimensionality reduction**: A technique used to reduce the number of features or dimensions in a dataset while preserving the most important information. (Definition) Dimensionality reduction is often used to visualize high-dimensional data, such as embeddings. (Explanation) For example, dimensionality reduction can be used to project a high-dimensional word embedding space into a lower-dimensional space, such as 2D or 3D, for visualization. (Example)

Other takeaways for readers:

* Embeddings are a powerful tool for representing complex data in a way that can be processed by neural networks.
* The choice of neural network architecture and training data can significantly impact the quality of the embeddings.
* Dimensionality reduction is a useful technique for visualizing and understanding high-dimensional embeddings.
* Embeddings can be used in a wide range of applications, including NLP, computer vision, and recommender systems.

AI 的回答, 是否和你亲自阅读得到的答案差不多呢? 可以看到, 细化后的 AI 总结其实不比原文短多少, 可见好处并非缩减篇幅, 而在于因为内容结构由你自己定义, 于是能够期待每段分别得到什么知识点｡

然后尝试 AI 对话｡ 以下是让 AI 在阅读 "Meaning Space and Semantic Laws of Motion" (Chapter 14) 时, 发生在 “初读者､ 专家､ 重读者” 三者间对话｡

**The Novice Reader**: Wow, I just finished reading this text and my mind is blown The idea that language can be represented as a feature space with coordinates is fascinating. I'm still trying to wrap my head around it, but it makes sense that semantically similar words would be placed nearby. I love the examples of how words are laid out in 2D, it's like a map of meaning. But what really caught my attention is the concept of "semantic laws of motion" - is it possible that there are underlying rules that govern how language works?

**The Expert Reader**: Ah, yes, the notion of a feature space is a fundamental concept in natural language processing. It's a way to quantify and visualize the relationships between words and their meanings. The idea of semantic laws of motion is indeed intriguing, and it's an area of ongoing research. One way to approach this is to think about the geometry of the feature space - are there certain patterns or structures that emerge when we analyze language in this way? The author's mention of "parallel transport" and "flatness" in the space is a great example of this. It's a complex topic, but it has the potential to reveal deep insights into the nature of language.

**The Returning Reader**: I remember reading this text for the first time and being struck by the idea that language can be represented in a mathematical space. But upon re-reading it, I'm struck by how much more nuanced and complex the concept is. The author's use of analogies to illustrate the relationships between words is brilliant, and it's amazing how well it works even when projected down to 2D. What I'm still grappling with is the idea that there might be a "semantic law of motion" that governs how language works. Is it a deterministic system, or is there room for creativity and innovation? And how do we balance the need for structure and rules with the need for flexibility and adaptability in language?

**The Novice Reader**: That's a great question I was wondering the same thing. If there are underlying rules that govern language, does that mean that our language is predetermined in some way? Or is it more like a set of guidelines that we can choose to follow or break? And what about the role of context and culture in shaping the meaning of words?

**The Expert Reader**: Ah, excellent questions The relationship between structure and creativity is a classic debate in linguistics. One way to think about it is that language is a complex system that is shaped by both universal principles and cultural-specific factors. The feature space representation can help us identify patterns and regularities, but it's also important to recognize that language is used by humans in a wide range of contexts, and that meaning is always negotiated and constructed in the moment. So, while there may be underlying rules or tendencies, they are always subject to variation and adaptation.

**The Returning Reader**: I think that's one of the things that I appreciate about this text - it's not trying to reduce language to a set of simple rules or formulas. Instead, it's acknowledging the complexity and messiness of language, while still trying to identify patterns and structures that can help us understand it better. And I love how the author is willing to admit that we may not have all the answers yet, and that there's still much to be discovered. It's a very humble and open-ended approach to language.

**The Novice Reader**: Yeah, I appreciate that too. It makes me feel like I'm not alone in my confusion, and that even experts are still grappling with these big questions. So, what's the next step in exploring these ideas? Are there any other texts or resources that you would recommend for someone who wants to dive deeper?

**The Expert Reader**: Ah, there are many resources available for exploring these ideas further. One classic text is "Foundations of Statistical Natural Language Processing" by Christopher Manning and Hinrich Schütze. It's a bit more technical, but it provides a comprehensive introduction to the mathematical and computational aspects of NLP. For a more accessible introduction, I would recommend "Language and the Mind" by Steven Pinker. It's a great overview of the cognitive and linguistic aspects of language.

**The Returning Reader**: And for those who want to explore the more philosophical and cultural aspects of language, I would recommend "The Meaning of Meaning" by C.K. Ogden and I.A. Richards. It's a classic text that explores the relationship between language, culture, and meaning.

上述讨论应该还不错｡ 后来又尝试让 mixtral-8x22b-instruct 用相同的内容举行三方会谈, 结论是 mixtral 不如 llama3-70b-instruct 讨论得深入､ 全面｡

AI三不能

AI 阅读效果不仅取决于模型的质量, 更取决于读者的引导｡ LLM 无法让讨厌读书的人读得更少, 只能帮助喜欢读书的人读得更多｡ 因为 AI

不能为你记住

总结越快, 忘记越快, 只因来得太便宜｡ 读过的书如果不写笔记, 不做习题, 不运用, GPT567来了也记不住｡

不能替你欣赏

隽永的诗歌, 睿智的对谈, 奇妙的小说, 精巧的论文; AI 帮你滤掉的水分, 恰恰是其精华｡

不能给你品味

甜还是咸, Rust 还是 PHP, 尼康还是佳能, 苹果还是华为｡ 好的 AI 恪守中庸, 一碗水端平; 正常的人应该有所偏爱｡

AI 不能替你读, 恰如真人不能替你读｡ 读者使用 AI 绝不是为了投机取巧或不劳而获, 而是为了把难读的变通顺, 把难懂的变易懂, 把不必细读的应付过去, 是为了更好地享受阅读的乐趣｡