Clio Music

• Year: 2007–2023
• Type: Patent & commercial technology

Overview

In 2007, I co-founded Orpheus Media Research and developed Clio Music — a machine-listening technology that analyzed music to automatically understand its harmonic structure, melodic content, rhythmic patterns, and emotional characteristics.

Unlike traditional music tagging systems that rely on human listeners to manually categorize music, Clio used computational analysis to extract musical features directly from audio recordings.

The technology was licensed exclusively to Rovi/TiVo/Xperi from 2012 to 2023 and is protected by U.S. Patent No. 8,084,677. It powered recommendation and discovery systems across tens of millions of tracks, predating the consumer wave of generative music AI by roughly a decade.

The Problem

Traditional music recommendation systems rely on:

• Manual tagging by human listeners — expensive, slow, subjective
• Collaborative filtering — requires large user bases, cold-start problem
• Metadata analysis — limited to what’s explicitly labeled

None of these approaches actually listen to the music itself.

The Solution

Clio analyzes audio recordings to automatically extract:

• Harmonic content: chord progressions, key changes, modulations
• Melodic patterns: contour, intervals, motifs
• Rhythmic structure: tempo, meter, syncopation
• Timbral characteristics: instrumentation, texture, density
• Emotional content: valence (happy/sad), arousal (calm/energetic)

This analysis happens entirely automatically, without human intervention.

Technical Approach

• Signal processing: FFT analysis, spectral decomposition
• Pattern recognition: machine learning classifiers
• Music theory models: harmonic analysis algorithms
• Perceptual models: psychoacoustic feature extraction

The result is a rich, multi-dimensional representation of each piece of music that can be used for similarity matching, recommendation, and discovery.

Applications

• Music recommendation systems (TiVo)
• Playlist generation
• Music search and discovery
• Automatic music categorization
• Forensic musicology (identifying musical similarities)

Patent

• U.S. Patent No. 8,084,677
• Title: “Methods and Systems for Music Analysis”
• Issued: December 27, 2011
• Assignee: Orpheus Media Research
• View patent on Google Patents

Publications & Presentations

• ISMIR — International Society for Music Information Retrieval
• Guest lectures at Vanderbilt, NYU, Drexel, and others

See also