How Generative AI Adds Value to the Future of Work
这篇Upwork的文章深入探讨了生成式人工智能（AI）在重新塑造工作价值方面的变革力量，强调了自动化和创新不仅改变了工作岗位，还在各个行业提高了生产力和创造力。文章着重讨论了对劳动力市场的细微影响，强调了技能发展和道德考虑的重要性，并对人工智能与人类合作的未来提供了前瞻性的视角。
Authors: Dr. Ted Liu, Carina Deng, Dr. Kelly Monahan
Generative AI’s impact on work: lessons from previous technology advancements
In this study, we provide a comprehensive analysis of the initial impact of generative AI (artificial intelligence) on the Upwork marketplace for independent talent. Evidence from previous technological innovations suggests that AI will have a dual impact: (1) the displacement effect, where job or task loss is initially more noticeable as technologies automate tasks, and (2) the reinstatement effect, where new jobs and tasks increase earnings over time as a result of the new technology. Take for example the entry of robotics within the manufacturing industry. When robotic arms were installed along assembly lines, they displaced some of the tasks that humans used to do. This was pronounced in tasks that were routine and easy to automate. However, new tasks were then needed with the introduction of robotics, such as programming the robots, analyzing data, building predictive models, and maintaining the physical robots. The effects of new technologies often counterbalance each other over time, giving way to many new jobs and tasks that weren’t possible or needed before. The manufacturing industry is now projected to have more jobs available as technologies continue to advance, including Internet of Things (IoT), augmented reality, and AI, which transform the way work is completed. The issue now at hand is ensuring enough skilled workers are able to work alongside these new technologies.
While this dynamic of displacement and reinstatement generally takes years to materialize, as noted above in the manufacturing example, the effects of generative AI may be taking place already on Upwork. For the platform as a whole, we observe that generative AI has increased the total number of job posts and the average spend per new contract created. In terms of work categories, generative AI has reduced demand in writing and translation, particularly in low-value work, while enhancing earnings in high-value work across all groups. In particular, work that relies on this new technology like Data Science and Analytics are reaping the benefits. The report highlights the importance of task complexity and the skill-biased nature of AI's impact. Skills-biased technology change is to be expected as the introduction of new technologies generally favors highly skilled workers. We observe this on our platform as high-skill freelancers in high-value work are benefiting more, while those in low-value work face challenges, underscoring the need for skilling and educational programs to empower freelancers to adapt and transition in this evolving work landscape.
Understanding the lifecycle of work on Upwork and the impact of gen AI
Generative AI has a growing presence in how people do their work, especially since the public release of ChatGPT in 2022. While there’s been extensive discussion about the challenges and opportunities of generative AI, there is limited evidence of such impact based on transaction data in the broader labor market. In this study, we use Upwork’s platform data to estimate the short-term effects of generative AI on freelance outcomes specifically. The advantage of the Upwork platform is that it is in itself a complete marketplace for independent talent, as we observe the full life cycle of work: job posts, matching, work execution, performance reviews, and payment. Few other instances exist where a closed-system work market can be studied and observed. Thus, the results of this study offer insights into not only the online freelance market, but also the broader labor market.
How technological progress disrupts the labor market is not a new topic. Acemoglu and Restrepo (2019) argue that earning gain arises from new tasks created by technological progress, which they term the “reinstatement effect,” even if the automation of certain tasks may have a displacement effect in the labor market initially. What this means is that there may be a dynamic effect going on: the displacement effect (e.g., work loss) may be more noticeable in the beginning of a new technology entry, but as new jobs and tasks are being created, the reinstatement effect (e.g., rates increase, new work) will begin to prevail. In the broader labor market, such dynamics will likely take years to materialize. But in a liquid and active independent work marketplace like Upwork, it’s possible that we’re already observing this transition happening.
Existing studies such as this provides a useful conceptual framework to think about the potential impact of generative AI. It’s likely that in the short term, the replacement of generative AI will continue to be more visible, not just at Upwork, but also in the broader labor market. Over time and across work categories, however, generative AI will likely spur new tasks and jobs, leading to the reinstatement effect becoming stronger and increasing rates for those occupations with new tasks and a higher degree of task complexity. We’ve already seen evidence of new demand as a result of gen AI on our Upwork platform, with brand new skill categories like AI content creator and prompt engineer emerging in late 2022 and early 2023. We test this hypothesis of both work displacement and reinstatement, and provide insights into how generative AI affects work outcomes.
Impact of generative AI on work
To understand the short-term impact of generative AI on the Upwork freelance market, we capitalize on a natural experiment arising from the public release of ChatGPT in November 2022. Because this release was largely an unanticipated event to the general public, we’re able to estimate the causal impact of generative AI. The essential idea behind this natural experiment is that we want to compare the work groups affected by AI with the counterfactual in which they are not. To implement this, we use a statistical and machine-learning method called synthetic control. Synthetic control allows us to see the impact that an intervention, in this case, the introduction of gen AI, has on a group over time by comparing it to a group with similar characteristics not exposed to the intervention. The advantage of this approach is that it allows us to construct reasonably credible comparison groups and observe the effect over time.
The units of analysis we use are work groups on the Upwork platform; we analyze variables such as contract number and freelancer earnings. Instead of narrowly focusing on a single category like writing, we extend the analysis to all the major work groups on Upwork. Moreover, we conduct additional analysis of the more granular clusters within each major group. The synthetic control method allows for flexibility in constructing counterfactuals at different levels of granularity. The advantage of our comprehensive approach is that we offer a balanced view of the impact of generative AI across the freelance market.
Generative AI’s short-term impact on job posts and freelancer earnings
Looking at the platform as a whole, we observe that generative AI has increased the total number of job posts by 2.4%, indicating the overall increased demand from clients. Moreover, as shown in Figure 1, for every new job contract, there is an increase of 1.3% in terms of freelancer earnings per contract, suggesting a higher value of contracts.
Figure 1 Effect of Generative AI on Freelancer Earning per Contract
The Upwork platform has three broad sectors: 1. Technological and digital solutions (tech solutions); 2. Creative & outreach; 3. Business operations and consulting. We have observed both positive and negative effects within each of the sectors, but two patterns are worth noting:
The reinstatement effect of generative AI seems to be driving growth in freelance earnings in sectors related to tech solutions and business operations. In contrast, within the creative sector, while sales and marketing earnings have grown because of AI, categories such as writing and translation seem disproportionately affected more by the replacement effect. This is to be expected due to the nature of tasks within these categories of work, where large language models are now able to efficiently process and generate text at scale.
Generative AI has propelled growth in high-value work across the sectors and may have depressed growth in low-value work. This supports a skills-biased technology change argument, which we’ve observed throughout modern work history.
More specifically and within tech solutions, data science & analytics is a clear winner, with over 8% of growth in freelance earnings attributed to generative AI. This makes sense as the reinstatement effect is at work; new work and tasks such as prompt engineering have been created and popularized because of generative AI. Simultaneously, while tools such as ChatGPT automate certain scripting tasks (therefore leading to a replacement effect), it mainly results in productivity enhancements for freelancers and potentially leads to them charging higher rates and enjoying higher overall earnings per task.
In terms of contracts related to business operations, we observe that accounting, administrative support, and legal services all experience gains in freelance earnings due to generative AI, ranging from 6% to 7%. In this sector, customer service is the only group that has experienced reduced earnings (-4%). The reduced earnings result for customer service contracts is an example of the aggregate earnings outcomes of AI, related to the study by Brynjolfsson et al (2023), who find that generative AI helps reduce case resolution time at service centers.
A potential outcome of this cut in resolution time is that service centers will need fewer workers, as more tasks can be completed by a person working alongside AI. At the same time, the reinstatement effect has not materialized yet because there are no new tasks being demanded in such settings. This may be an instance where work transformation has not yet been fully realized, with AI enabling faster work rather than reinventing a way of working that leads to new types of tasks. A contrasting case is the transformation that happened with bank tellers when ATMs were introduced. While the introduction of these new technologies resulted in predictions of obsolete roles in banks, something different happened over time. Banks were able to increase efficiency as a result of ATMs and were able to scale and open more branches than before, thereby creating more jobs. In addition, the transactional role of a bank teller became focused on greater interpersonal skills and customer relationship tasks.
When taken together, the overall gains in such business operations work on Upwork are an encouraging sign. These positions tend to require relatively intensive interpersonal communication, and it seems the short-term effects of generative AI have helped increase the value of these contracts, similar to what we saw in the banking industry when ATMs were introduced.
As of now, the replacement effect of AI seems more noticeable in creative and outreach work. The exception is sales and marketing contracts, which have experienced a 6.5% increase in freelance earnings. There is no significant impact yet observed on design. For writing and translation, however, generative AI seems to have reduced earnings by 8% and 10% respectively. However, as we will discover, task complexity has a moderating effect on this.
High-value work benefit from generative AI, upskilling needed for low-value work
Having discussed the overall impact of generative AI across categories, we now decompose the impact by values. The reason we’re looking at the dimension of work value is that there may be a positive correlation between contract value and skill complexity. Moreover, skill complexity may also be positively correlated with skill levels. Essentially, by evaluating the impact of AI by different contract values, we can get at the question of AI's impact by skill levels. This objective is further underscored by a discrepancy that sometimes exists in the broader labor markets – a skills gap between demand and supply. It simply takes time for upskilling to take place, so it’s typical for demand to exceed supply until a more balanced skilled labor market takes place. It is worth noting, however, freelancers on the Upwork platform seem more likely than non-freelancers to acquire new skills such as generative AI.
For simplicity, let’s assume that the value of contracts is a good proxy for the level of skill required to complete them. We’d then assume that high-skill freelancers typically do high-value work, and low-skill freelancers do low-value work. In other words, our goal is also to understand whether the impact of generative AI is skills-biased and follows a similar pattern from what we’ve seen in the past with new technology disruptions. Note that we’re focusing on the top and bottom tails of the distribution of contract values, because such groups (rather than median or mean) might be most susceptible to displacement and/or reinstatement effects, therefore of primary concern. We define high-value (HV) work as those with $1,000 or more earnings per contract. For the remaining contracts, we focus on a subset of work as low-value (LV) work ($251-500 earnings).
Figure 2 shows the impact of AI by work value, across groups on Upwork. As we discussed before, writing and translation work has experienced some reduction in earnings overall. However, if we look further into the effect of contract value, we see that the reduction is largely coming from the reduced earnings from low-value work. At the same time, for these two types, generative AI has induced substantial growth in high-value earnings – the effect for translation is as high as 7%. We believe the positive effect on translation high-value earning is driven by more posts and contracts created. In the tech solutions sector, the growth in HV earnings in data science and web development is also particularly noticeable, ranging from 6% to 9%. Within the business solutions sector, administrative support is the clear winner.
There are two takeaways from this analysis by work value. First, while we’re looking at a sample of all the contracts on the platform, it’s possible that the decline of LV work is more than made up for by the growth of HV work in the majority of the groups. In other words, except for select work groups, the equilibrium results for the Upwork freelance market overall seem to be net positive gains from generative AI. Second, if we assume that freelancers with high skills (or a high degree of skill complexity) tend to complete such HV work (and low-skill freelancers do LV work), we observe that the impact of generative AI may be biased against low-skill freelancers. This is an important result: In the current discussion of whether generative AI is skill-based, there exists limited evidence based on realized gains and actual work market transactions. We are one of the first to provide market-transaction-based evidence to illustrate this potentially skill-biased impact. Finally, additional internal Upwork analysis finds that independent talent engaged in AI-related work earn 40% more on the Upwork marketplace than their counterparts engaged in non-AI-related work. This suggests there may be additional overlap between high-skill work and AI-related work, which can further reinforce the earning potential of freelancers in this group.
Figure 2
Case study: 3D content work
To illustrate the impact of generative AI in more depth, we have conducted a case study of Engineering & Architecture work within the tech solutions sector. The reason is that we want to illustrate the potentially overlooked aspects of AI impact, compared with the examples of data science and writing contracts. This progress in generative AI has the potential to reshape work in traditional areas like design in manufacturing and architecture, which rely heavily on computer-aided design (CAD) objects, and newer sectors such as gaming and virtual reality, exemplified by NVIDIA's Omniverse.
Based on activities on the Upwork platform, we see that there is consistent growth of job posts and client spending in this category, with up to 12% of gross service value growth year over year in 2023 Q3, and over 11% in job posts during the same period. Moreover, applying the synthetic control method, we show a causal relationship between gen AI advancements and the growth in job posts and earnings per contract. More specifically, there is a significant increase in overall earnings because of AI, an average 11.5% increase. Additionally, as shown by Figure 3, the positive effect also applies to earning per contract. This indicates a positive impact on freelancer productivity and quality of work, due to the fact that we’re measuring the income for every unit of work produced. This suggests that gen AI is not just a facilitator of efficiency but also enhances the quality of output.
Figure 3 Effect of Generative AI on Freelancer Earning per Contract in EngineeringIn a traditional workflow to create 3D objects without generative AI, freelancers would spend extensive time and effort to design the topology, geometry, and textures of the objects. But with generative AI, they can do so through text prompts to train models and generate 3D content. For example, this blog by NVIDIA’s Omniverse team showcases how ChatGPT can interface with traditional 3D creation tools.
Thus, the positive trajectory of generative AI in 3D content generation we see is driven by several factors. AI significantly reduces job execution time, allowing for higher productivity. It facilitates the replication and scaling of 3D objects, leading to economies of scale. Moreover, freelancers can now concentrate more on the creative aspects of 3D content, as AI automates time-consuming and tedious tasks.
This shift has not led to a decrease in rates due to the replacement effect. In fact, this shift of workflow may create new tasks and work. We will likely see a new type of occupation in which technology and humanities disciplines converge. For instance, a freelancer trained in art history now has the tools to recreate a 3D rendering of Japan in the Edo period, without the need to conduct heavy coding. In other words, the reinstatement effect of AI will elevate the overall quality and value proposition of the work, and ultimately enable higher earning gains. This paradigm shift underscores generative AI's role in not just transforming work processes but also in creating new economic dynamics within the 3D content market. Fortunately, it seems many freelancers on Upwork are ready to reap the benefits: 3D-related skills, such as 3D modeling, rendering, and design, are listed among the top five skills of freelancer profiles as well as in job posts.
A dynamic interplay: task complexity, skills, and gen AI
Focusing on the Upwork marketplace for independent talent, we study the impact of generative AI by using the public release of ChatGPT as a natural experiment. The results suggest a dynamic interplay of replacement and reinstatement effects; we argue that this dynamic is influenced by task complexity, suggesting a skill-biased impact of gen AI. Analysis across Upwork's work sectors shows varied effects: growth in freelance earnings in tech solutions and business operations, but a mixed impact in the creative sector. Specifically, high-value work in data science and business operations see significant earnings growth, while creative contracts like writing and translation experience a decrease in earnings, particularly in lower-value tasks. Using the case study of 3D content creation, we show that generative AI can significantly enhance productivity and quality of work, leading to economic gains and a shift toward higher-value tasks, despite initial concerns of displacement.
Acemoglu and Restrepo (2019) argue that the slowdown of earning growth in the United States the past three decades can partly be explained by new technologies’ replacement effect overpowering the reinstatement effect. But with generative AI, we’re at a point of completely redefining what human tasks mean, and there may be ample opportunities to create new tasks and work. It's evident that while high-value types of work are being created, freelancers engaged in low-value tasks may face negative impact, possibly due to a lack of skills needed to capitalize on AI benefits. This situation underscores the necessity of supporting freelancers not only in elevating their marketability within their current domains but also in transitioning to other work categories.
To ensure as many people as possible benefit, there’s an imperative need to provide educational resources for them to gain the technical skills, and more importantly skills of adaptability to reinvent their work. This helps minimize the chance of missed opportunities by limiting skills mismatch between talent and new demands created by new technologies. Upwork has played a significant role here by linking freelancers to resources such as Upwork Academy’s AI Education Library and Education Marketplace, thereby equipping them with the necessary tools and knowledge to adapt and thrive in an AI-present job market. This approach can help bridge the gap between low- and high-value work opportunities, ensuring a more equitable distribution of the advantages brought about by generative AI.
Methodology
To estimate the causal impact of generative AI, we take a synthetic control approach in the spirit of Abadie, Diamond, and Hainmueller (2010). The synthetic control method allows us to construct a weighted combination of comparison units from available data to create a counterfactual scenario, simulating what would have happened in the absence of the intervention. We use this quasi-experimental method due to the infeasibility of conducting a controlled large-scale experiment. Additionally, we use Lasso regularization to credibly construct the donor pool that serves the basis of the counterfactuals and minimize the chance of overfitting the data.
Moreover, we supplement the analysis by scoring whether a sub-occupation is impacted or unaffected by generative AI. The scoring utilizes specific criteria: 1. Whether a certain share of job posts are tagged as AI contracts by the Upwork platform; 2. AI occupational exposure score, based on a study by Felten, Raj, and Seamans (2023), to tag these sub-occupations. We also use data smoothing techniques through three-month moving averages. We analyzed data collected on our platform from 2021 through Q3 2023. We specifically look at freelancer data across all 12 work categories on the platform for high-value contracts, defined as those with a contract of at least $1,000, and low-value contracts, consisting of those between $251 and under $500.
The main advantage of our approach is that it is a robust yet flexible way to identify the causal effects on not only the Upwork freelance market but also specific work categories. Additionally, we control for macroeconomic or aggregate shocks such as U.S. monetary policy in the pre-treatment period. However, we acknowledge the potential biases in identifying which sub-occupations are influenced by generative AI and the effects of external factors in the post-treatment period.
About the Upwork Research Institute
The Upwork Research Institute is committed to studying the fundamental shifts in the workforce and providing business leaders with the tools and insights they need to navigate the here and now while preparing their organization for the future. Using our proprietary platform data, global survey research, partnerships, and academic collaborations, we produce evidence-based insights to create the blueprint for the new way of work.
About Ted Liu
Dr. Ted Liu is Research Manager at Upwork, where he focuses on how work and skills evolve in relation to technological progress such as artificial intelligence. He received his PhD in economics from the University of California, Santa Cruz.
About Carina Deng
Carian Deng is the Lead Analyst in Strategic Analytics at Upwork, where she specializes in uncovering data insights through advanced statistical methodologies. She holds a Master's degree in Data Science from George Washington University.
About Kelly Monahan
Dr. Kelly Monahan is Managing Director of the Upwork Research Institute, leading our future of work research program. Her research has been recognized and published in both applied and academic journals, including MIT Sloan Management Review and the Journal of Strategic Management.
Economic Impact
2024年02月23日
Economic Impact
糟糕的客户体验导致 2024 年全球销售额 3.7 万亿美元面临风险
This analysis by the XM Institute highlights the profound impact of poor customer experiences on global sales, projecting a risk of $3.7 trillion in 2024. Through a comprehensive study across 26 countries involving over 28,000 respondents, it unveils that bad customer interactions predominantly in sectors like government, fast food, and automotive can lead to significant financial losses. The report further delineates how consumer spending patterns shift post negative experiences, emphasizing the critical need for companies to prioritize customer satisfaction to safeguard their revenues.
作为我们最新的全球消费者研究的一部分，Qualtrics XM Institute 询问了 26 个国家/地区的 28,000 多人对最近的不良经历的反应。然后我们计算了他们改变的消费行为如何影响不同的行业。我们的分析表明：
最令人失望的是政府。消费者表示，最常在政府机构（26%）遇到糟糕的客户体验，而在超市（6%）则最不常遇到。
快餐的销量损失最大。在经历了一次糟糕的经历后，消费者最有可能停止或减少在快餐店的支出，并且最不可能减少与公共事业的互动（64%）。
汽车经销商流失的客户最多。在经历了一次糟糕的经历后，消费者最有可能完全停止与汽车经销商 (22%) 开展业务，最不可能停止与公用事业 (6%) 和超市 (6%) 开展业务。
25 个国家的糟糕体验的代价
根据本研究和世界银行的数据，我们估计所研究的 25 个国家/地区的公司正将高达 28 亿美元的销售额置于风险之中1。为了估计这个数字，我们测量了：
不良经历的频率。14% 的消费者表示与他们交往的公司有不好的经历。在各个国家，这一比例从日本的 6% 到印度的 26% 不等。
一次糟糕的经历后，支出会发生变化。38% 的不良体验后，消费者会减少支出，而另外 13% 的负面体验后，消费者会完全停止在公司的支出。在各个国家，减少或停止在公司支出的比例从新西兰的 37% 到法国的 63% 不等。
面临风险的销售（百分比）。当我们将不良体验的百分比与削减或停止支出的消费者的百分比相乘时，我们最终得到“面临风险的销售”的总体水平为 7%。在各个国家，“风险销售”的比例从日本的 2% 到印度和泰国的 11% 不等。
销售面临风险（货币）。为了将百分比转换为货币，我们将每个国家/地区的风险销售额百分比乘以世界银行的家庭消费数据（以美元为单位）。在 25 个国家/地区，我们发现总计 2.8 万亿美元的销售额因不良体验而面临风险，其中芬兰的 50 亿美元到美国的 8460 亿美元不等
全球影响：3.7 万亿美元面临风险
在我们估算了 25 个国家/地区面临风险的销售后，我们研究了全球范围内不良体验的影响。为此，我们再次查阅了世界银行的数据。事实证明，我们 25 个国家的家庭消费占全球的 78%。如果我们假设世界其他地区的风险销售数据与我们 25 个国家/地区的数据相当，那么我们会发现 2024 年有 3.7 万亿美元的销售面临不良经历的风险。总计：
消费者将停止在提供极差客户体验的公司上花费 1 万亿美元。
消费者将减少 2.7 万亿美元在提供极差客户体验的公司上的支出。
那么发生了什么变化？
一年多前，我们估计全球有 3.1 万亿美元因不良体验而面临风险。 2024 年有什么不同？我们发现消费者是：
花费更多。自我们之前的分析发布以来，全球家庭消费支出总额跃升超过 7.7 万亿美元 — — 这意味着更多的收入可能面临风险。
报告较少的不良经历。与去年相比，消费者报告遇到不良客户体验的频率减少了 2.2 个百分点。航空公司（-10 个百分点）、财产保险公司（-8 个百分点）和大学（-8 个百分点）的不良体验下降幅度最大。
对糟糕的经历做出更严重的反应。在这两年接受调查的 20 个行业中，因互动不良而导致消费者停止或减少支出的比例增加了 1.6 个百分点。增幅最大的是快餐（+5 个百分点），其次是包裹递送服务、汽车经销商和航空公司（+4 个百分点）。
底线：糟糕的体验伴随着（不断增长的）成本。
Talia Quaadgras是 XM Institute 的研究项目经理
Bruce Temkin，XMP，CCXP，Qualtrics XM 研究所负责人
我们在 2023 年全球消费者研究中减去了 5 个国家，并添加了两个不同的国家，其中包括 26 个国家，全球共有 28,400 名受访者。由于样本量较小，我们今年的分析中没有包括香港。
Economic Impact
Economic Impact
Economic Impact
扫一扫
添加客服
