▎原文地址
由人工智能驱动的推荐引擎对用户行为具有重大影响,并可能导致改变生活的决策。
关于推荐算法中**“好”的概念是主观的**,取决于各个利益相关者的观点。
算法本质上是编辑性的,反映了其创建者的价值观和选择。
产品设计在决定推荐引擎的有效性和影响方面起着至关重要的作用。
货币化策略显著影响推荐引擎的优化和部署方式。
文本以Megan和Jake的两个对比故事开头,他们都因YouTube的推荐引擎而经历了重大生活变化。这一介绍为讨论由人工智能驱动的推荐及其对个人和社会的影响奠定了基础。
作者强调,与人工智能推荐相关的**“好”的概念取决于利益相关者**。不同的群体(宗教组织、无神论者、YouTube、社会、内容生产者、政府、广告商)可能对什么是正面结果有不同的看法。
文本强调了推荐引擎在各个平台上的广泛使用,包括YouTube、电子商务网站和电子邮件服务。它指出,人工智能显著增强了这些算法的力量,使它们在塑造用户体验和选择方面更具影响力。
作者表达了对人工智能驱动的推荐引擎的伦理影响的担忧,特别是最近人工智能热潮的背景下。文本承认围绕这些算法是否有害、中立或有益的持续辩论。
作者提出了三条规则来指导关于人工智能推荐引擎的讨论:
算法是深度编辑性的,塑造意见。
理解编辑性:科技公司需要认识到,算法的设计和训练数据的选择本质上是编辑性的决定。这意味着公司在选择哪些数据输入、哪些数据排除时,实际上是在做出价值判断。
透明度和责任:公司应提高透明度,公开其算法的设计原则和数据来源,以便外界能够理解和监督这些决定。例如,Instagram联合创始人Kevin Systrom在讨论其新闻推荐应用Artifact时提到,构建算法是一个编辑性的过程,因为选择训练算法的数据和目标函数都是编辑性的判断。
设计设定了影响的边界。
用户体验优化:公司应仔细设计产品,以确保推荐算法能够根据用户的真实需求和行为提供有价值的内容。这包括理解用户的输入信号和数据优先级。例如,Facebook在其推荐引擎中面临的挑战表明,设计不当可能导致负面内容的泛滥。
平衡利益相关者需求:公司需要在设计中平衡不同利益相关者的需求,避免单一目标(如最大化用户参与度)导致的负面后果。内部备忘录显示,Facebook的推荐算法在优化参与度时,往往会推动负面内容的传播。
货币化决定了公司使用这些算法的激励方向。
多样化货币化策略:公司应考虑多种货币化策略的利弊,避免单一的广告模式带来的负面影响。例如,广告模式可能会导致用户花费更多时间在平台上,但也可能助长错误信息和愤怒情绪。
用户控制和选择:尽可能赋予用户更多的控制权,让他们能够调整推荐算法以符合个人偏好。然而,作者也指出,这种方法可能需要用户具备高水平的技术知识,因此公司应提供简化和易于理解的控制选项。
文本探讨了科技公司的价值观和选择如何影响推荐引擎的设计和实施,以YouTube为例。它强调,技术并不是在一个无道德的真空中创造的,而是反映了个人的观点和公司的价值观。
这一部分讨论了产品设计如何影响推荐引擎的性能和结果。作者将Facebook在推荐系统方面的挣扎与TikTok的算法友好设计进行了比较。
Facebook 在推荐系统方面的挣扎
负面内容的泛滥:Facebook 的推荐系统被发现偏向于推送负面内容。根据公司的内部研究,数据输入(如点赞或分享)倾向于显示负面内容。这导致了平台上负面言论的泛滥。
透明度和责任感缺失:Facebook 的推荐系统缺乏透明度,外界难以理解其算法的工作原理。尽管投入了大量资金,推荐引擎仍无法满足所有用户的需求。
设计不当:Facebook 的推荐引擎在优化用户参与度时,往往会推动负面内容的传播。内部备忘录显示,平台的核心产品机制(如病毒传播、推荐和优化参与度)是负面言论泛滥的主要原因。
难以平衡社会和企业需求:Facebook 面临的一个主要挑战是如何在实现企业增长的同时,平衡更广泛的社会利益。2019 年的一份内部备忘录指出,平台的机制并非中立,如果不采取措施,Facebook 将无意中促进负面活动。
TikTok 的算法友好设计
快速反馈机制:TikTok 的视频格式较短,允许用户快速反馈。视频自动播放,用户通过观看或滑动来表达对推荐内容的喜好。这种设计使得算法能够迅速了解用户偏好。
机器学习标签:TikTok 通过机器学习标签(如视频内容包含小狗、滑雪等)来了解用户的兴趣,从而更精准地推荐内容。
成瘾性设计:TikTok 的设计使其成为一种高度成瘾的体验,被称为“科技界的毒品”。这种设计不仅提高了用户的参与度,还使其成为全球成功的消费类社交应用之一。
透明度和用户体验:相比 Facebook,TikTok 更加公开其推荐算法的设计原则和数据来源。通过用户反馈不断优化算法,提升用户体验。
文本概述了三种主要的货币化策略(广告模式、单元销售和订阅)及其对推荐引擎设计和结果的影响。它强调,所选择的货币化方法可以显著影响系统的相对“好”。
作者承认创建一个满足所有利益相关者并真正赋予用户权力的完美推荐引擎的难度。文本还讨论了由于互联网信息量巨大,禁止这些系统的不现实性。
文章总结了理解和改进人工智能推荐系统的重要性。作者宣布了一系列即将发布的文章,这些文章将深入探讨推荐系统中的技术方面、商业模式和人工智能内容生成可能引发的颠覆性变化。
Megan was brought into the embrace of the good lord Jesus Christ through the power of YouTube. She started with mommy bloggers who had, as she described it, a “really positive energy.” From there, she noticed that they frequented the same Targets and drank Diet Coke from the same drive-throughs and had the same bleached blonde hair and went to the same church—i.e., they were all from Utah.
梅根通过 YouTube 的力量被带入了耶稣基督的怀抱。她从一些妈妈博主开始,这些博主有着她所描述的“非常积极的能量”。从那里,她注意到她们常去同样的 Target 商店,从同样的得来速买饮料喝同样的健怡可乐,有着同样的漂白金发,并去同样的教堂——也就是说,她们都来自犹他州。
Her investigation into their personal lives surfaced a video series entitled “I’m a Mormon.” She dove into the deep end of the baptismal font (metaphorically speaking), watching dozens of hours of sermons on YouTube. Eventually, she requested a Book of Mormon to be dropped off at her house. I would know, I was the zitty 20-year-old missionary YouTube put on her doorstep to deliver it. Shortly thereafter, she got dunked in a baptismal font (not metaphorically speaking) and joined the LDS Church. On that day, she reported feeling “hopeful and free for the first time in a long time.”
她对他们个人生活的调查发现了一系列名为“我是摩门教徒。”的视频。她全身心投入到洗礼池(比喻意义上),在YouTube上观看了数十小时的布道视频。最终,她请求有人把《摩尔门经》送到她家。我知道,因为我是那个油腻腻的20岁传教士,YouTube把我送到她家门口递交它。不久之后,她在洗礼池中(不是比喻意义上)受了洗,并加入了耶稣基督后期圣徒教会。那天,她报告说“第一次感到希望和自由,已经很久没有这种感觉了。”
Jake escaped the grips of the same organization through YouTube. He had recently returned home from a mission to a far-off country and was watching the same “I’m a Mormon” videos. The system then recommended a new series: “I’m an Ex-Mormon.” Jake was sucked in—dozens of hours of videos were consumed. From there, Google directed him to various blogs where people questioned the tenets of the faith he had just spent two years preaching. After several years of questioning and doubting, he left the LDS church. I should know, Jake is my friend. When I asked him how he felt after leaving, he reported, “Hopeful and free for the first time in a long time.” Note: Both names have been changed to protect privacy.
杰克通过YouTube逃脱了同一个组织的掌控。他最近刚刚完成了一个前往遥远国家的任务,回到家中观看相同的“我是摩门教徒”的视频。系统随后推荐了一个新系列:“我是前摩门教徒。”杰克被吸引住了——观看了数十小时的视频。从那里,谷歌引导他进入了各种博客,人们在这些博客中质疑他刚刚传教两年的信仰教义。经过几年的质疑和怀疑,他离开了摩门教。我应该知道,杰克是我的朋友。当我问他离开后的感受时,他回答说:“第一次感到充满希望和自由。” 注意:为了保护隐私,名字均已更改。
You may or may not like religion, but that is irrelevant. What matters is this: Did the AI recommendation do good? The emotional outcome was identical for the individuals. To the best of my knowledge, neither person regrets the choice they made. And, still, neither person would’ve made the change they did without YouTube’s recommendation engine surfacing just the right video at just the right time.
你可能喜欢或不喜欢宗教,但这无关紧要。重要的是:AI推荐是否做了好事?对个人来说,情感结果是相同的。据我所知,没有人后悔他们所做的选择。而且,如果没有YouTube的推荐引擎在恰当的时间出现恰当的视频,这两个人都不会做出他们所做的改变。
The challenge is that “good” is stakeholder dependent. If you’re a devout Mormon, Jake’s choice was bad, potentially dooming his soul. If you’re a committed atheist, Megan was a fool, suckered into a cult. In either case, YouTube finds both outcomes good because the two consumed dozens of hours of ad-supported videos before making this decision. Other stakeholders—like society at large, content producers, governments, or advertisers—may have different perspectives on the relative good of YouTube’s AI-powered conversions.
挑战在于“好”是依赖于利益相关者的。如果你是虔诚的摩门教徒,杰克的选择是坏的,可能会让他的灵魂陷入危险。如果你是坚定的无神论者,梅根是个傻瓜,被吸进了一个邪教组织。无论哪种情况,YouTube 都认为这两种结果是好的,因为两人在做出这个决定之前,观看了数十小时的广告支持视频。其他利益相关者——如整个社会、内容生产者、政府或广告商——可能对 YouTube 的 AI 驱动转换的相对好坏有不同的看法。
To further muddy the waters, how much good is even attributable to YouTube is debatable. Ask yourself: What percentage of these two individuals’ actions can be credited to the information they received versus their own free will? To what degree do you believe in individual agency?
进一步使情况变得复杂的是,YouTube 所带来的好处有多少也是值得商榷的。问问自己:这两个人的行为中有多少可以归因于他们从中获得的信息,而多少是由于他们自己的自由意志?你在多大程度上相信个人的能动性?
This isn’t some mere philosophical debate. Over one billion hours of video are consumed by YouTube’s users every day. Over 70% of the videos consumed are surfaced by algorithmic feeds. It is the second most visited website in the world. And the beating heart of its success is a recommendation engine.
这不仅仅是一些哲学辩论。YouTube 用户每天观看的视频超过 十亿 小时。超过 70% 的视频是通过算法推荐的。它是世界上访问量第二大的网站。而其成功的核心是推荐引擎。
Recommendation engines, sometimes called recommendation algorithms, have been blamed for Trump’s election, Biden’s election, mass shootings, the proliferation of vegan restaurants, and TikTok’s dominance. The tech is responsible for you reading this very article. Whether Gmail put this in your “main” inbox, spam, or social tab, the destination was determined by some type of recommendation engine. They permeate e-commerce sites and travel websites. Anywhere there is more information than the eye can scan, algorithms are at work surfacing results.
推荐引擎,有时被称为推荐算法,被指责为特朗普当选、拜登当选、大规模枪击事件、素食餐厅的激增以及TikTok的主导地位的原因。这项技术使你正在阅读这篇文章。无论Gmail将其放在你的“主要”收件箱、垃圾邮件或社交标签中,目的地都是由某种类型的推荐引擎确定的。它们渗透到电子商务网站和旅游网站。只要有眼睛无法扫描的过多信息,算法就会在背后工作以提供结果。
AI has made that math far more potent. The same scientific advances powering products like ChatGPT or autonomous vehicles are also telling you to watch the new trailer for a Marvel movie. These algorithms don’t operate in a vacuum. They are battling head to head, formula to formula, vying for dominance in the market for eyeballs. I wrote about this phenomenon last May, arguing that “addiction is becoming the blood sacrifice required of consumers to allow businesses to win.” In the war zone of the internet, the most time spent wins.
人工智能使得这种数学变得更加强大。驱动像ChatGPT或自动驾驶汽车这样的产品的同样科学进步也在告诉你去观看一部漫威电影的新预告片。这些算法不是在真空中运行的。它们在市场上为了吸引眼球而展开激烈的竞争,公式对公式,争夺主导地位。我去年五月写过这个现象,认为“上瘾正在成为消费者为了让企业获胜所需的血祭。”在互联网的战场上,花费时间最多的就能获胜。
In the meeting of these two concerns, of commercial interests and ethical conduct, the recent AI boom has me concerned. While we are all still debating the ethical implications of this technology, the algorithms keep getting better. Services are becoming more addicting and there isn’t much an individual can do about it. Tech companies will often defend recommendation engines by pointing out that when they are deployed, use time and customer ratings increase, thus proving that these algorithms are “good.” This is a circular argument—of course these things go up when the customer has access to them, that is what they are designed to do.
在商业利益与道德行为这两种关注点的交汇处,最近的人工智能热潮让我感到担忧。虽然我们仍在讨论这项技术的伦理影响,但算法变得越来越好。服务变得越来越令人上瘾,个人对此无能为力。科技公司经常通过指出当推荐引擎被部署时,使用时间和客户评分都会增加,从而证明这些算法是“好的。”来为推荐引擎辩护。这是一个循环论证——当然,当客户可以使用这些东西时,这些数据会增加,因为这正是它们的设计目的。
It feels like everyone has an opinion on this tech. Some people believe that algorithms are terrible and cause harm, while others believe that they are morally neutral or even helpful in giving people what they want. However, in my opinion, this debate is far too black-and-white and oversimplified. Instead, it would be more productive to examine how algorithms impact our decision-making and culture by considering three rules that are often overlooked by one or both sides.
感觉每个人对这项技术都有自己的看法。有些人认为算法很糟糕并且会造成伤害,而另一些人则认为它们是道德中立的,甚至有助于满足人们的需求。然而,在我看来,这场辩论过于非黑即白且过于简化。相反,更有成效的方法是通过考虑经常被一方或双方忽视的三条规则来研究算法如何影响我们的决策和文化。
Tech companies need to acknowledge that algorithms are deeply editorial, meaning they have the power to shape opinions and perspectives.
科技公司需要承认,算法是深具编辑性的,这意味着它们有能力塑造观点和看法。
Critics must recognize that design sets the boundaries of influence.
批评者必须认识到,设计设定了影响力的界限。
Third, monetization sets the boundaries for what companies are incentivized to do with these algorithms.
第三,货币化设定了公司在这些算法上受到激励去做什么的界限。
By examining these overlooked factors, we can have more nuanced discussions about the role of AI amplification in society.
通过审视这些被忽视的因素,我们可以就AI放大在社会中的作用展开更细致的讨论。
Think of these rules as playing a similar role to what physics does during a game of basketball—gravity doesn’t determine who wins the game, but it does determine if the ball goes in the hoop. My three rules don’t determine which system is “good,” but they do allow us to understand how that goodness comes to be.
把这些规则想象成在篮球比赛中物理学所起的作用——重力不会决定谁赢得比赛,但它确实决定了球是否进篮。我这三条规则并不决定哪个系统是“好”的,但它们确实让我们理解那种好是如何产生的。
Now is the time to figure this out. Society stands in the center of an AI cage match. In the left-hand corner, are incredibly powerful AI recommendation engines, and in the right, are new Generative AI tools that are exponentially increasing the volume of content to filter. If we don’t get this right, the past few years of misinformation, election interference, and false panics will look quaint by comparison.
现在是解决这个问题的时候了。 社会正处于人工智能笼子赛的中心。在左边角落,是强大的人工智能推荐引擎,在右边角落,是成倍增加内容量的新生成式人工智能工具。如果我们搞不定这个问题,过去几年的错误信息、选举干扰和虚假恐慌相比之下会显得微不足道。
As we discussed earlier, there is no universal “good” when it comes to recommendation engines because you have to balance stakeholder priorities. Editorial algorithms are one such way this phenomena manifests itself—the creators of a product get to determine what values they want to protect or dismantle in their app, by choosing what types of content and topics users will see. Technology isn’t created in an amoral vacuum. It is personal opinion forced upon the world. To see it in practice, we return to YouTube.
正如我们之前讨论的那样,当涉及到推荐引擎时,没有普遍的“好”,因为你必须平衡利益相关者的优先事项。编辑算法就是这种现象的一种表现方式——产品的创造者可以通过选择用户将看到的内容类型和主题来决定他们想在应用程序中保护或拆除哪些价值观。技术并不是在道德真空中创造的。它是强加于世界的个人意见。为了看到它的实际运作,我们回到YouTube。
When the service was in its infancy, it had two simultaneous methods of recommending features. The first, a delightfully analog attempt, was a team of “cool hunters” whose job was to scour the website for good videos to feature on the homepage. This was paired with a simple algorithm that recommended related videos based on co-visitation; e.g., if you liked this video, another user liked one just like it. But even in those early days, there were editorial choices made with the algorithm’s design. For example, early recommendation experiments in the autos and vehicles category just surfaced a bunch of fetish videos of women’s feet revving engines in luxury cars. The choice was made by the cool hunter in charge of cars to designate this as “bad.” While a group of rather horny dudes who loved cars and feet might disagree with that call, the broader population was probably pleased. Around the same time, another tweak to the system accidentally made the related sections full of ”boobs and asses basically.”
当服务还处于初期阶段时,它有两种同时进行的推荐功能的方法。第一个是一个令人愉快的模拟尝试,那就是一个“酷猎手”团队,他们的工作是浏览网站,寻找值得在主页上展示的优质视频。这与一个简单的算法相结合,该算法基于共同访问推荐相关视频;例如,如果你喜欢这个视频,另一位用户也喜欢一个类似的视频。但即使在那些早期阶段,算法设计也有编辑选择。例如,早期在汽车和车辆类别中的推荐实验只显示了一堆女性用豪华车踩油门的恋物视频。负责汽车的酷猎手决定这类视频为“不好”。虽然一群非常喜欢汽车和脚的色狼可能不同意这个决定,但更广泛的群体可能会感到满意。大约在同一时间,对系统的另一个调整意外地使相关部分充满了“胸部和屁股基本上。”
For the first six years of YouTube, the recommendations mostly focused on optimizing videos that got clicks. The results were predictable—quick videos with catchy titles did great. However, in 2012, YouTube shifted the recommendation engine from primarily rewarding clicks to rewarding watch time. Now, rather than rewarding the videos with the most clickable thumbnails, the system would reward creators who had longer videos that people finished. This shift devastated all the creators who had built their editorial brand around being short and punchy. This change caused an initial 25% drop in revenue for YouTube.
在YouTube的前六年,推荐系统主要集中在优化获得点击的视频。结果是可预测的——快速的视频和吸引人的标题表现很好。然而,在2012年,YouTube将推荐引擎从主要奖励点击量转向奖励观看时间。现在,系统不再奖励那些拥有最具点击性缩略图的视频,而是奖励那些拥有较长视频且观众能够看完的创作者。这一转变对所有以简短和生动为特色的创作品牌造成了毁灭性的影响。这一变化导致了YouTube最初收入下降25%。
By 2017, the platform was using an AI technique called reinforcement learning to recommend videos to viewers. The company made over 300 tweaks to the system that year alone, with the goal of getting the recommendation just right. Its theory was the better the video recommendations, the more time people would spend watching YouTube because they’d get sucked back in. It worked. 70% of YouTube views in 2017 came from recommended videos, and overall views increased by 1%.
到2017年,该平台使用了一种称为强化学习的AI技术向观众推荐视频。公司在这一年对系统进行了超过300次调整,目的是让推荐更精准。其理论是,视频推荐越好,人们在YouTube上花费的时间就越多,因为他们会被吸引回来。这种方法奏效了。2017年,YouTube 70%的观看量来自推荐视频,总观看量增加了1%。
However, even with the supposedly neutral AI running the show, editorial choices were made. In one instance, YouTube decided that an excessive amount of videos it deemed “gross” was being displayed on the homepage. To tackle this issue, the company adjusted the algorithm, utilizing a computer model called the “trash video classifier.” Even the names of the programs have value statements.
然而,即便是在所谓中立的人工智能控制下,编辑选择仍然存在。在一个实例中,YouTube 认为首页上显示了过多它认为“恶心”的视频。为了解决这个问题,公司调整了算法,使用了一种称为“垃圾视频分类器”的计算机模型。甚至这些程序的名称都带有价值判断。
At every decision, with every type of algorithm, Google’s ethical views are fed into the code, determining what videos see the light of day. Measuring engagement as a way to determine video quality, and then maximizing that, is an editorial choice with moral consequences. Each recommendation proffered to a user is a gentle form of persuasion, one that imbues certain values. The lack of transparency regarding the values coded in these recommendation systems obstructs our ability to understand what is happening—and the only reason we know any of this is because of years of dedicated journalism. The company has never been transparent about how it all works.
在每一个决策中,每一种算法中,谷歌的伦理观点都被输入到代码中,决定了哪些视频能够见到天日。将参与度作为确定视频质量的方法,然后最大化这一点,是一种具有道德后果的编辑选择。每个推荐给用户的内容都是一种温和的说服形式,赋予了特定的价值观。这些推荐系统中编码价值观缺乏透明性,妨碍了我们理解所发生的事情的能力——我们之所以知道这一切,完全是因为多年来的专注新闻报道。该公司从未对其运作方式透明过。
I like the way Instagram co-founder Kevin Systrom put it when discussing the recent launch of his news recommendation app Artifact. “[B]uilding the algorithm is enormously editorial. Because what you choose to train your algorithm on—the objective function, the data you put in, the data you include, the data you don’t include—is all in editorial judgment,” he said.
我喜欢Instagram联合创始人Kevin Systrom在讨论最近推出他的新闻推荐应用Artifact时的说法。“[构建算法是一个巨大的编辑过程。因为你选择用什么来训练你的算法——目标函数,你输入的数据,你包含的数据,你不包含的数据——这都是编辑判断,”他说。
When we consider if an AI system is good, we must recognize the value system that the creators are imposing on it.
当我们考虑一个人工智能系统是否良好时,我们必须认识到创造者所强加的价值体系。
Just as values shape the algorithms’ performance, so does the design of the product in which the engine is housed. To understand the good or bad of an AI recommendation, you need to understand what data inputs and user signals that an app prioritizes.
正如价值观塑造了算法的性能,产品的设计也同样影响引擎的表现。要了解AI推荐的优劣,您需要了解应用程序优先考虑的数据输入和用户信号。
No company has had a longer history of facing public criticism of their recommendation engine than Facebook has. It feels as if Mark Zuckerberg is called in front of Congress every other week to testify about something. No matter how much money was thrown at it, the recommendation engine could never really get to the point of satisfying everyone. An internal Facebook memo written in 2019, entitled “What Is Collateral Damage?” offers some clues as to why the platform was never able to balance a broader social good with its corporate growth demands:
没有哪家公司比 Facebook 更长时间地面对公众对其推荐引擎的批评。感觉马克·扎克伯格每隔一周就会被召到国会作证。无论投入了多少钱,推荐引擎似乎从未真正达到让所有人满意的程度。2019 年 Facebook 内部的一份备忘录,题为“什么是附带损害?”提供了一些线索,解释了为什么该平台从未能平衡更广泛的社会利益与其企业增长需求:
“We also have compelling evidence that our core product mechanics, such as virality, recommendations, and optimizing for engagement, are a significant part of why these types of [negative] speech flourish on the platform.
“我们也有令人信服的证据表明,我们核心产品机制(如病毒性、推荐和优化参与度)是这些类型的[负面]言论在平台上盛行的重要原因。
“If integrity takes a hands-off stance for these problems, whether for technical (precision) or philosophical reasons, then the net result is that Facebook, taken as a whole, will be actively (if not necessarily consciously) promoting these types of [negative] activities. The mechanics of our platform are not neutral.”
“如果在这些问题上保持不干涉的立场,无论是出于技术(精度)还是哲学原因,那么最终结果是,作为一个整体的Facebook将积极(即使不一定是有意识地)推动这些类型的[负面]活动。我们平台的机制并不是中立的。”
According to the company’s own research, the data inputs for how Facebook makes recommendations, such as likes or shares, were biased toward surfacing negative content. The Facebook files further showed that the company had followed the path of YouTube. They’d notice a problem and make some changes, hoping that the outcome would be OK without necessarily understanding the black box of the algorithm. However, both companies operate at the scale of the entire human race. It isn’t possible for them to make any changes without causing significant harm somewhere.
根据公司的自身研究,Facebook 推荐内容的数据输入(如点赞或分享)存在偏向,容易显示负面内容。Facebook 文件进一步显示,公司遵循了 YouTube 的路径。他们注意到一个问题并进行了一些更改,希望结果会好,但不一定能理解算法的黑箱。然而,这两家公司都在全球范围内运营。不可能在不引起某些地方重大损害的情况下进行任何更改。
On the other end of the spectrum, TikTok’s entire app design is entirely focused on improving its recommendation engine. The format of the videos (short) allows for rapid feedback. The video autoplays, meaning that users will immediately show their feelings about the suggestion by watching or swiping. By combining this with machine learning labels (this video has puppies, skiing, etc.), the algorithm can know what you like. Eugene Wei calls this “algorithm-friendly design.”
在光谱的另一端,TikTok 的整个应用设计完全专注于改进其推荐引擎。视频的格式(短视频)允许快速反馈。视频自动播放,这意味着用户会通过观看或滑动立即显示他们对建议的感受。通过将其与机器学习标签(此视频包含小狗、滑雪等)结合起来,算法可以知道你喜欢什么。Eugene Wei 称其为“算法友好设计”。
The outcome is that TikTok created the most addicting experience I’ve ever had in technology. It is TV on crack. It has become one of the few consumer social apps to achieve global success in the past few years. It is so powerful that it has convinced teenagers in the U.S. that they have Tourette syndrome. One stakeholder (TikTok shareholders) would be ecstatic. The kid’s parents? Not so much.
结果是,TikTok 创造了我在科技中体验过的最令人上瘾的体验。它是比电视更上瘾。在过去的几年里,它成为了少数几个在全球取得成功的消费类社交应用之一。它如此强大,以至于让美国的青少年相信他们有妥瑞氏症综合症。一个利益相关者(TikTok 股东)会欣喜若狂。孩子的父母?不见得。
Compare this to Facebook, where the recommendation engine was tacked on two years after the company launched via the invention of the News Feed. Because the input signals were less clear, Zuckerberg had a tough time understanding what was happening, meaning the company had a harder time fine-tuning the performance. When we consider whether a system is good, we have to look at how the design shapes the outcomes.
相比之下,Facebook 的推荐引擎是在公司通过新闻推送的发明推出后两年才附加上的。由于输入信号不太明确,扎克伯格很难理解发生了什么,这意味着公司在微调性能方面更加困难。当我们考虑一个系统是否优秀时,我们必须看看设计如何塑造结果。
Charlie Munger once said, “Show me the incentive and I will show you the outcome.” When evaluating a recommendation engine, how the company monetizes will affect the relative good. There are three monetization strategies, each with their own trade-offs and costs:
查理·芒格曾经说过:“给我看激励措施,我会告诉你结果。” 在评估推荐引擎时,公司如何盈利将影响相对的好处。有三种盈利策略,每种都有其权衡和成本:
The current bogeyman is ad models. Companies will deploy AI recommendations to incentivize people to scroll, swipe, and watch for longer. It is rare that someone says, “I am so glad I spent an extra 15 minutes on TikTok today,” but that is what the system is optimized for. The downsides are fairly obvious—at their extreme, ad models can foster misinformation and outrage. Prominent companies like Meta, Snapchat, Twitter, and Pinterest all tune their recommendation engines for this use case.
当前的“恶棍”是广告模式。公司将部署AI推荐系统,以激励人们更长时间地滚动、滑动和观看。很少有人会说:“我很高兴今天在TikTok上多花了15分钟”,但这正是系统所优化的目标。其缺点显而易见——在极端情况下,广告模式可能会助长错误信息和愤怒。像Meta、Snapchat、Twitter和Pinterest这样的知名公司都调整了他们的推荐引擎以适应这种用例。
The second most common use case for recommendation engines is discrete unit sales. Think about when you buy a product on Amazon and it asks, “Do you also want this?” It is useful, but it also increases the average order value and “makes” people spend more money. Once again, the good of this model is relative. It is good for Amazon, good for the small businesses that sell on the platform, but bad for compulsive shoppers. Sometimes you’ll see these engines not immediately monetize but try to keep people engaged for the chance to sell other products later on. Platforms that monetize via unit sales this way include Airbnb, Etsy, and Walmart.com.
推荐引擎的第二个最常见的使用案例是离散单元销售。想想当你在亚马逊上购买一件产品时,它会问:“你也想要这个吗?” 这很有用,但它也增加了平均订单价值并“让”人们花更多的钱。再次强调,这种模式的好处是相对的。它对亚马逊有利,对在平台上销售的小企业有利,但对冲动购物者不利。有时你会看到这些引擎不会立即货币化,而是试图让人们保持参与,以便以后有机会销售其他产品。通过这种方式通过单元销售来货币化的平台包括Airbnb、Etsy和Walmart.com。
Finally, there are subscriptions. In this case, recommendation systems are designed to offer a stunted product experience, with embedded cliffhangers that make people pull out their credit cards. Think of Tinder—it deliberately designs the experience to be tantalizing and horny and frustrating all at once. When users are hooked, they are asked to get a subscription to boost their profile and (supposedly) improve their outcomes. Other companies like Netflix, Substack, and The New York Times are compelled to paywall their most hookable content, so when you want the service most you have to pull your credit card out to continue.
最后是订阅。在这种情况下,推荐系统的设计是为了提供一种受限的产品体验,其中嵌入了悬念,使人们不得不掏出他们的信用卡。想想Tinder——它故意设计出一种既诱人又让人感到欲望和挫败的体验。当用户上钩时,他们被要求订阅以提升他们的个人资料并(据称)改善他们的结果。其他公司如Netflix、Substack和《纽约时报》被迫对他们最具吸引力的内容设置付费墙,所以当你最想要这项服务时,你必须掏出信用卡才能继续。
Importantly, all of these systems can be construed as having “bad” outcomes. You can also make an equally compelling argument that these monetization schemes are good! Ads make information free, sales don’t burden people with repeat bills, subscriptions foster long-term consumer/creator relationships. Unfortunately, it isn’t as obvious as ads suck.
重要的是,所有这些系统都可以被认为有“坏”结果。 你也可以提出一个同样有力的论点,即这些货币化方案是好的!广告让信息免费,销售不让人负担重复账单,订阅促进长期的消费者/创作者关系。不幸的是,这并不像广告糟糕那么明显。
To make it even more complicated, almost all companies will offer some combination of all three of these methods. Shoot, this very publication has a subscription tier, sells ad slots, and offers educational courses you can buy. I am not immune from criticism along any of these dimensions. My interaction with recommendation engines outside of the occasional tweet or inbox sorting is minimal, but I do think about it a lot!
更复杂的是,几乎所有公司都会提供这三种方法的某种组合。说真的,这本出版物本身就有一个订阅层次,出售广告位,并提供可以购买的教育课程。我在这些方面都不能免于批评。我与推荐引擎的互动,除了偶尔的推文或收件箱排序之外,几乎没有,但我确实经常考虑这个问题!
What actually matters is that all three of these monetization methods are competing in the exact same marketplace—the internet. The winner of a respective market is competing on which method best suits their competitive landscape, not what suits stakeholder good. When we are trying to say whether X or Y system is relatively good, what negative and positive behavior is incentivized must be considered.
真正重要的是,这三种货币化方法都在同一个市场——互联网中竞争。各自市场的赢家在于哪种方法最适合其竞争环境,而不是适合利益相关者的利益。当我们试图说X或Y系统相对较好时,必须考虑激励了哪些负面和正面的行为。
Imagine the perfect recommendation engine.
想象一下完美的推荐引擎。
Everyone who uses it is magically inspired to maximize their utility. Suddenly the world is full of people with washboard abs, PhDs, and ever-present smiles. People change their actions because they got the information they needed at the exact time they needed it. This is clearly a fantasy! It will never happen. The gulf between what we say versus what we actually do is huge, colossal, enormous. There is a difference between people’s stated versus revealed preferences.
每个使用它的人都被神奇地激发去最大化他们的效用。突然间,世界上充满了拥有洗衣板般腹肌、博士学位和永远微笑的人们。人们改变了他们的行为,因为他们在需要的时候得到了所需的信息。这显然是一个幻想!这永远不会发生。我们所说的与我们实际做的之间有巨大的鸿沟,巨大,庞大,庞大。人们所说的偏好和实际表现的偏好之间是有区别的。
I just don’t think it is possible to get a recommendation system that makes everyone happy; there are too many stakeholders with directly contradictory needs.
我只是觉得不可能有一个推荐系统能让所有人都满意;有太多利益相关者的需求是直接矛盾的。
The answer could potentially be that companies should empower users to control the recommendation engines they interact with. This sounds tempting, but I have concerns. We don’t interact with one or two of these engines a day—there are dozens. They are all around us, everpresent, always watching. How likely is it that users are able to correctly individually tune each of these? Real control over these systems would require a technical knowledge of neural networks, weights, and data sets 99.99% of consumers don’t comprehend. Shoot, the very companies that built these things don’t truly understand what happens inside them! Why should users be expected to figure it out? To say that user-empowerment fixes this issue feels like token empowerment versus true individual freedom.
答案可能是公司应该授权用户控制他们互动的推荐引擎。这个想法很有诱惑力,但我有一些担忧。我们每天与这些引擎互动的不止一两个——有几十个。它们无处不在,无时无刻不在观察。用户有多大可能能够正确地单独调整每一个呢?对这些系统的真正控制需要对神经网络、权重和数据集的技术知识,而99.99%的消费者都不理解。事实上,构建这些东西的公司本身也不完全理解它们的内部运作!为什么要用户去弄明白呢?说用户赋权能解决这个问题感觉像是象征性的赋权,而不是真正的个人自由。
The other supposedly obvious answer would be to just ban them. However, it isn’t that simple. There is simply too much data, too much information on the internet, for us to abolish the techniques. To make the internet usable, we have to have recommendation engines. A world without them is a world with less choice, worse information, and boring entertainment.
另一个看似显而易见的答案是直接禁止它们。然而,事情并没有那么简单。互联网上的数据和信息实在是太多了,无法废除这些技术。为了使互联网可用,我们必须拥有推荐引擎。一个没有它们的世界是一个选择更少、信息更差、娱乐更乏味的世界。
Frankly, right now, I don’t know the best answer. The whole point of this essay is to show that there is not a pithy solution to this issue. It requires serious study and hard work for all of us, as technologists, to figure out. I hope that my three frameworks below can allow for us to examine this topic with a more discerning eye:
坦率地说,现在我不知道最好的答案。这篇文章的重点是表明这个问题没有一个简洁的解决方案。作为技术专家,我们都需要认真研究和努力工作才能找出答案。我希望我下面的三个框架可以让我们以更敏锐的眼光来审视这个话题:
Algorithms are editorial
算法是编辑
Design determines outcomes
设计决定结果
Monetization doesn’t care about your feelings
货币化不在乎你的感受
Over the next month, I’ll be examining this topic in detail. It really, really matters. The internet has given me my career, introduced me to my wife, and given me empathy for people like Jake and Megan. I want it to thrive and improve and continue to be one of humanity’s crowning achievements. To keep it and to protect it, this next decade of builders will have to do better.
在接下来的一个月里,我将详细研究这个主题。这真的非常非常重要。互联网给了我我的事业,让我认识了我的妻子,并让我对像杰克和梅根这样的人产生了同情。我希望它能够蓬勃发展,改善并继续成为人类的辉煌成就之一。为了保持和保护它,下一代的建设者必须做得更好。
This essay is the first in a series about AI and recommendation systems that I’ll be writing over the next few weeks. It will cover:
这篇文章是我在接下来的几周内将要写的一系列关于人工智能和推荐系统的第一篇。它将涵盖:
The AI math and techniques that make this all possible
使这一切成为可能的人工智能数学和技术
The history and development of business models that make this technology profitable
使这项技术盈利的商业模式的历史和发展
How AI content generation breaks the whole system
人工智能内容生成如何打破整个系统