The Data Annotation Business Behind AI Music

11/19 2026 525

Header image generated by AI

By | Tian Yu

Ed. | Shen Xiao

To learn music, one must first understand rhythm, chords, and instruments. The same holds true for AI.

In October, an AI-generated song titled "Come Before" went viral. Historical and mythological figures—Du Fu, Su Shi, Liu Bang—rapped about mortgages, office life, and midlife struggles, striking a chord with many listeners. Others were blunt: it sounded like AI from the first note, and it made them uncomfortable.

AI has also made its way into professional music creation. Jiangsu TV's latest season of "Music Fate Plan 3" centers on originality, with its selling point being the stories behind the original works. Yet in the first episode, a musician admitted to using AI in the production—though he could vouch that the lyrics were his own.

After listening, Xue Zhiqian remarked: "I can understand the lyrics, yet they feel strangely familiar." Many viewers believed the lyrics were a patchwork of several of Xue Zhiqian's songs.

The song itself is not unpleasant to listen to. The controversy lies elsewhere: a program built around original value featured a song whose creation relied heavily on AI—so on what grounds should it compete on the same stage as other original works?

Many are also curious: how does AI learn to write songs? Wang Kunzhe, the creator of "Come Before," did not hand the entire creative process over to AI. He first built the skeleton of the lyrics, then let AI try different combinations, generating multiple musical versions, after which he screened, edited, and adjusted the arrangement himself.

AI can generate music, but that does not mean it truly understands music. To teach a model to recognize the structure of a song and judge whether the generated result is good or bad, a group of people who understand music is still needed behind the scenes.

They are not responsible for writing songs; they act as music teachers for AI. These people are AI music annotators.

In 2018, Suno was still four years away from being born, and AI-generated music was even further from becoming a hot topic.

Liu Xiaoguang had already begun pondering a question: how to make machines understand music.

This entrepreneurial direction is somewhat counterintuitive. The most exciting stories in AI startups usually involve letting machines create something. What Liu Xiaoguang was doing was exactly the opposite: he first had to teach machines to recognize music.

Liu Xiaoguang holds bachelor's, master's, and doctoral degrees from the Department of Chemistry at Tsinghua University and is also a music producer. The company he founded, DeepMusic, initially targeted music data processing.

Hubei Daily recently reported on Wuhan-based music data service company Yaosheng Shuzhi. According to its head, Huang Binyao, annotators must first listen for the key and tempo, determine the harmonic progression, then use tools to measure the beat, divide the song into different sections, and identify the instrumentation section by section.

For example, this section uses violin, while another features a blended timbre of brass and woodwinds, with strings in the background. Whether the lyrics are sung incorrectly, whether the instruments are used appropriately, and whether the rhythm matches may also become information that needs to be labeled.

This work is music annotation.

Strictly speaking, these are a group of professional positions that emerged around AI music training and evaluation, and the boundaries are not entirely fixed.

Even before generative AI appeared, music platforms needed to tag massive numbers of songs. The pop, sad, rock, and other categories we see in music apps all rely on music data organization behind the scenes. These tags mainly served song search, classification, and recommendation.

After the rise of generative music tools such as Suno and Udio, what models need to learn became more complex: in addition to knowing what style a song belongs to, they also need to understand its chords, rhythm, instrumentation, and structure, and even judge whether the music they generate has problems.

As a result, music annotation gradually developed into several different types of work.

Some people are responsible for breaking down basic information; some specialize in judging whether the melody, lyrics, and arrangement are reasonable; and others categorize errors and organize them into feedback that model R&D teams can use. Someone needs to tell AI exactly what is wrong with the music it generates.

Overseas, the U.S. AI talent platform micro1 has also begun recruiting music teachers, performers, and producers to participate in model training, with hourly wages listed at $30 to $45.

Beyond such high pay, at present in China, for basic annotation and internship positions of this kind, a monthly salary of a few thousand yuan is the common level.

In 2023, Liu Xiaoguang mentioned that large enterprises were purchasing data from them for music generation and music recognition.

Tencent Music is an important strategic investor in DeepMusic. DeepMusic integrated AI arrangement technology into Quanmin K Song and launched an AI style-switching feature. At that time, Quanmin K Song users used AI accompaniment more than 300 million times. Today, DeepMusic's official website lists this as an important case of technology implementation; Quanmin K Song's AI layout continues to expand.

Behind music annotation, several different businesses have also developed.

The first is selling labor. This is the most basic business and the most direct source of employment for music annotators.

If an AI company needs professionally annotated data for 100,000 songs but does not have enough music professionals of its own, hiring a dedicated music team is clearly not cost-effective. Thus third-party data service providers emerged.

Service providers are responsible for recruiting annotators, setting work standards, assigning tasks, checking quality, and finally delivering the organized data to clients. AI companies do not need to maintain a music team long term; they only need to purchase services when there is a project. This is very similar to traditional data annotation outsourcing.

The second is selling professional judgment. This is more valuable than simple annotation.

When an AI company is preparing to release a new version of a music model, it needs to know whether the new model has actually improved, so it has to find professional musicians to conduct blind listening tests.

The same set of lyrics is used to have two models generate songs separately, and then musicians score them across dimensions such as melody fluency, arrangement rationality, vocal performance, and style matching. After identifying problems, they analyze whether the error lies in melody generation or in the model's failure to understand the creative instructions.

What is ultimately delivered to the client is a set of evaluation results, error analysis, and optimization suggestions.

This has already become a formal part of AI music R&D. The 2025 ICASSP conference included a study called MusicEval. The research team collected 2,748 pieces of music generated by 31 AI music models, invited 14 music experts to score them, and ultimately obtained 13,740 evaluations.

The third is selling data assets.

If a company simultaneously has annotation capabilities, owns a large amount of music material, and has also resolved the corresponding licensing issues, it can package audio, music structure, professional tags, text descriptions, and usage rights and license them to AI companies for model training.

Overseas, Rightsify's GCX platform, which specializes in AI music data licensing, licenses music datasets that have been organized, annotated, and cleared for corresponding rights to AI R&D enterprises and research institutions. GCX's dataset scale has exceeded 4.4 million hours of audio and 32 billion sets of text data, with total capacity exceeding 3 PB.

Its pricing is quoted based on data scale, complexity, usage scenarios, and licensing term. GCX also emphasizes that the datasets obtain corresponding rights through methods such as commissioned creation and provide commercial licensing.

The imaginative space of this business is clearly much greater than simply selling labor.

But problems follow: where does this music come from?

For major record companies, unauthorized music being used to train models means damage to copyright and commercial interests; they still have strong resources to respond. For ordinary musicians, they often cannot bear the consequences.

An original musician in Shandong revealed that he had once received a call from an AI sampling company hoping to collect his original music for model training. Other musicians have posted new works in exchange groups, only for them to be intercepted, trained with AI, and regenerated in the blink of an eye. Many people have filed lawsuits over this.

Works created through painstaking effort have become nourishment for AI training, while AI's output may not bring any benefit to the original creators.

There is a paradox in this business: the harder people work to teach AI to understand music, the more opportunity AI has to learn to listen on its own.

The UMP Board developed by DeepMusic began trying to use technology to automatically extract music information long ago. By 2023, this system was already able to autonomously complete automatic annotation of 400,000 songs.

But early automation was still mostly about handling standardized information on people's behalf. The real change has occurred in the past two years.

On May 1 this year, MOSI.AI, the OpenMOSS team, and Shanghai Innovation Institute jointly open-sourced MOSS-Music, including two 8B models and a supporting large-scale music data annotation pipeline.

The pipeline supporting this model can already start from raw audio and automatically complete feature extraction, song segmentation, lyric recognition, metadata cleaning, and then generate music descriptions and training samples. Even the evaluation of music descriptions is beginning to be attempted by using another large model to score them.

What people once had to listen to over and over with headphones, machines can now complete a considerable portion of on their behalf.

However, music AI does not have only one technical route.

One route is to break music down into recognizable structured information, such as the aforementioned DeepMusic's UMP system and MOSS-Music.

The other route is end-to-end music generation represented by products such as Suno. Users input a piece of text, and the model directly generates complete music.

The difference between the two routes determines that demand for music annotators cannot keep growing in the same direction forever. The former requires more refined music data, while the latter may reduce reliance on manual annotation song by song.

But what they have in common is that both require data quality control and evaluation of generated results.

Looking at the entire AI music market, the speed at which machines generate music has already far exceeded the speed at which humans consume music.

Data released by overseas music platform Deezer shows that in June this year, the platform received an average of 90,000 AI-generated songs per day, accounting for more than half of daily new uploads at peak times.

By October, Deezer further disclosed that from January 2025 to August 2026, the platform had excluded more than 42 million AI-generated songs from algorithmic and editorial recommendations.

Despite such a massive number of AI songs, their actual plays accounted for only 1% to 3% of the platform's total plays. Of the vast amount of content produced, how much is worth hearing?

For music annotators, the focus of future work will also change: basic recognition will be handed over to machines, while professional musicians will participate more in setting evaluation standards, judging complex issues, and correcting model errors.

Looking across the entire music field, from annotating music for AI to helping AI judge good from bad, and then worrying that one's own works will become nourishment for AI training, the relationship between musicians and AI is far more complex than simple replacement versus being replaced.

When Zhou Shen released his new song "Chronicle of the Moon," he specifically stated that it is strictly prohibited to use the work for AI training, imitation, learning, and generation without permission.

Algorithms can replicate melodies and imitate styles, but that does not mean they possess the life experience of creators; models can produce music in bulk, but they cannot produce cultural depth.

AI has opened up unprecedented vast horizons for music. What kind of music is worth preserving, and how creators should receive the value they deserve, ultimately still need to be answered by people. The core of art is forever rooted in people.

Solemnly declare: the copyright of this article belongs to the original author. The reprinted article is only for the purpose of spreading more information. If the author's information is marked incorrectly, please contact us immediately to modify or delete it. Thank you.