Skip to content

Add semantic tests for the MusicXML export #199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wang1zhen wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add semantic tests for the MusicXML export #199

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6,641 changes: 6,641 additions & 0 deletions tests/musicxml-4.0/musicxml.xsd
View file
Open in desktop

Large diffs are not rendered by default.

57 changes: 57 additions & 0 deletions tests/musicxml-4.0/xlink.xsd
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema targetNamespace="http://www.w3.org/1999/xlink" elementFormDefault="qualified" attributeFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xlink="http://www.w3.org/1999/xlink">
<xs:annotation>
<xs:documentation>MusicXML XLink module

Version 4.0

Copyright © 2004-2021 the Contributors to the MusicXML Specification, published by the W3C Music Notation Community Group under the W3C Community Final Specification Agreement (FSA):

https://www.w3.org/community/about/agreements/final/

A human-readable summary is available:

https://www.w3.org/community/about/agreements/fsa-deed/

This is the W3C XML Schema (XSD) version of the MusicXML 4.0 language. Validation is tightened by moving MusicXML definitions from comments into schema data types and definitions. Character entities and other entity usages that are not supported in W3C XML Schema have also been removed. However, the features of W3C XML Schema make it easier to define variations of the MusicXML format, either via extension or restriction.

This schema module defines the subset of XLink attributes that are supported in the MusicXML schema. All definitions here are in the XLink namespace.</xs:documentation>
</xs:annotation>

<xs:attribute name="href" type="xs:anyURI"/>

<xs:attribute name="type">
<xs:simpleType>
<xs:restriction base="xs:NMTOKEN">
<xs:enumeration value="simple"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>

<xs:attribute name="role" type="xs:token"/>

<xs:attribute name="title" type="xs:token"/>

<xs:attribute name="show">
<xs:simpleType>
<xs:restriction base="xs:NMTOKEN">
<xs:enumeration value="new"/>
<xs:enumeration value="replace"/>
<xs:enumeration value="embed"/>
<xs:enumeration value="other"/>
<xs:enumeration value="none"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>

<xs:attribute name="actuate">
<xs:simpleType>
<xs:restriction base="xs:NMTOKEN">
<xs:enumeration value="onRequest"/>
<xs:enumeration value="onLoad"/>
<xs:enumeration value="other"/>
<xs:enumeration value="none"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:schema>
145 changes: 145 additions & 0 deletions tests/musicxml-4.0/xml.xsd
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
<?xml version='1.0'?>
<xs:schema targetNamespace="http://www.w3.org/XML/1998/namespace" xmlns:xs="http://www.w3.org/2001/XMLSchema" xml:lang="en">

<xs:annotation>
<xs:documentation>
See http://www.w3.org/XML/1998/namespace.html and
http://www.w3.org/TR/REC-xml for information about this namespace.

This schema document describes the XML namespace, in a form
suitable for import by other schema documents.

Note that local names in this namespace are intended to be defined
only by the World Wide Web Consortium or its subgroups. The
following names are currently defined in this namespace and should
not be used with conflicting semantics by any Working Group,
specification, or document instance:

base (as an attribute name): denotes an attribute whose value
provides a URI to be used as the base for interpreting any
relative URIs in the scope of the element on which it
appears; its value is inherited. This name is reserved
by virtue of its definition in the XML Base specification.

id (as an attribute name): denotes an attribute whose value
should be interpreted as if declared to be of type ID.
This name is reserved by virtue of its definition in the
xml:id specification.

lang (as an attribute name): denotes an attribute whose value
is a language code for the natural language of the content of
any element; its value is inherited. This name is reserved
by virtue of its definition in the XML specification.

space (as an attribute name): denotes an attribute whose
value is a keyword indicating what whitespace processing
discipline is intended for the content of the element; its
value is inherited. This name is reserved by virtue of its
definition in the XML specification.

Father (in any context at all): denotes Jon Bosak, the chair of
the original XML Working Group. This name is reserved by
the following decision of the W3C XML Plenary and
XML Coordination groups:

In appreciation for his vision, leadership and dedication
the W3C XML Plenary on this 10th day of February, 2000
reserves for Jon Bosak in perpetuity the XML name
xml:Father
</xs:documentation>
</xs:annotation>

<xs:annotation>
<xs:documentation>This schema defines attributes and an attribute group
suitable for use by
schemas wishing to allow xml:base, xml:lang, xml:space or xml:id
attributes on elements they define.

To enable this, such a schema must import this schema
for the XML namespace, e.g. as follows:
&lt;schema . . .>
. . .
&lt;import namespace="http://www.w3.org/XML/1998/namespace"
schemaLocation="http://www.w3.org/2001/xml.xsd"/>

Subsequently, qualified reference to any of the attributes
or the group defined below will have the desired effect, e.g.

&lt;type . . .>
. . .
&lt;attributeGroup ref="xml:specialAttrs"/>

will define a type which will schema-validate an instance
element with any of those attributes</xs:documentation>
</xs:annotation>

<xs:annotation>
<xs:documentation>In keeping with the XML Schema WG's standard versioning
policy, this schema document will persist at
http://www.w3.org/2007/08/xml.xsd.
At the date of issue it can also be found at
http://www.w3.org/2001/xml.xsd.
The schema document at that URI may however change in the future,
in order to remain compatible with the latest version of XML Schema
itself, or with the XML namespace itself. In other words, if the XML
Schema or XML namespaces change, the version of this document at
http://www.w3.org/2001/xml.xsd will change
accordingly; the version at
http://www.w3.org/2007/08/xml.xsd will not change.
</xs:documentation>
</xs:annotation>

<xs:attribute name="lang">
<xs:annotation>
<xs:documentation>Attempting to install the relevant ISO 2- and 3-letter
codes as the enumerated possible values is probably never
going to be a realistic possibility. See
RFC 3066 at http://www.ietf.org/rfc/rfc3066.txt and the IANA registry
at http://www.iana.org/assignments/lang-tag-apps.htm for
further information.

The union allows for the 'un-declaration' of xml:lang with
the empty string.</xs:documentation>
</xs:annotation>
<xs:simpleType>
<xs:union memberTypes="xs:language">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value=""/>
</xs:restriction>
</xs:simpleType>
</xs:union>
</xs:simpleType>
</xs:attribute>

<xs:attribute name="space">
<xs:simpleType>
<xs:restriction base="xs:NCName">
<xs:enumeration value="default"/>
<xs:enumeration value="preserve"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>

<xs:attribute name="base" type="xs:anyURI">
<xs:annotation>
<xs:documentation>See http://www.w3.org/TR/xmlbase/ for
information about this attribute.</xs:documentation>
</xs:annotation>
</xs:attribute>

<xs:attribute name="id" type="xs:ID">
<xs:annotation>
<xs:documentation>See http://www.w3.org/TR/xml-id/ for
information about this attribute.</xs:documentation>
</xs:annotation>
</xs:attribute>

<xs:attributeGroup name="specialAttrs">
<xs:attribute ref="xml:base"/>
<xs:attribute ref="xml:lang"/>
<xs:attribute ref="xml:space"/>
<xs:attribute ref="xml:id"/>
</xs:attributeGroup>

</xs:schema>
73 changes: 73 additions & 0 deletions tests/musicxml_corpus.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
"""A corpus of LilyPond snippets that must convert to valid MusicXML."""
import pytest


CORPUS = {
'melody_relative': r"""\relative c'' { \clef treble \key g \major \time 3/4 g4 a b | c2 b4 | a2. \bar "|." }""",
'piano_staff': r"""\score { \new PianoStaff << \new Staff = "up" \relative c'' { \clef treble c4 d e f | g1 } \new Staff = "down" { \clef bass c2 g, | c1 } >> }""",
'choir_staff': r"""\score { \new ChoirStaff << \new Staff { \set Staff.instrumentName = "S" c''1 } \new Staff { \set Staff.instrumentName = "A" a'1 } >> }""",
'staff_group': r"""\score { \new StaffGroup << \new Staff { c''1 } \new Staff { \clef bass c1 } >> }""",
'header': r"""\header { title = "Title" composer = "Composer" copyright = "CC" } \score { { c'1 } }""",
'header_markup': r"""\header { title = \markup { \bold "Title" } } { c'1 }""",
'tuplets': r"""{ \tuplet 3/2 { c'8 d' e' } \times 2/3 { f'4 g' a' } \tuplet 3/2 4 { b'8 c'' d'' e'' f'' g'' } }""",
'tuplet_span': r"""{ \tupletSpan 4 \tuplet 3/2 { c'8 d' e' f' g' a' } c''2 }""",
'grace_types': r"""{ \grace c'16 d'4 \acciaccatura e'8 f'4 \appoggiatura g'8 a'4 \slashedGrace b'8 c''4 }""",
'grace_chord': r"""{ \grace <c' e'>16 d'4 e' f' g' }""",
'ties_slurs': r"""{ c'4~ c' d'( e') | f'\( g' a' b'\) | c''1 }""",
'articulations': r"""{ c'4-. d'-- e'-> f'-^ | g'-_ a'-! b'\staccatissimo c''\fermata }""",
'ornaments': r"""{ c''4\trill d''\turn e''\mordent f''\prall }""",
'dynamics': r"""{ c'4\pp d'\< e' f'\! | g'\ff\> a' b' c''\p | c''1\sfz }""",
'dynamics_text': r"""{ c'4\cresc d' e' f'\! | g'1 }""",
'fingering': r"""{ c'4-1 d'-2 e'-3 f'-4 }""",
'chord_repetition': r"""{ <c' e' g'>4 q q q | <d' f'>2 q }""",
'isolated_durations': r"""{ c'4 4 8 8 4 }""",
'rhythmic_staff': r"""\new RhythmicStaff { 4 4 8 8 4 }""",
'drums': r"""\new DrumStaff \drummode { bd4 sn bd sn | hh8 hh hh hh hh hh hh hh }""",
'multi_rest': r"""{ \compressMMRests R1*3 | c'1 }""",
'skips': r"""{ s2 c'2 | s1 | d'1 }""",
'clefs': r"""{ \clef treble c'4 \clef alto c' \clef tenor c' \clef bass c | \clef "treble_8" c1 }""",
'keys_modes': r"""{ \key d \major d'1 | \key f \minor f'1 | \key e \dorian e'1 }""",
'time_changes': r"""{ \time 2/4 c'2 | \time 6/8 c'4. c'4. | \time 3/2 c'1. }""",
'numeric_time': r"""{ \numericTimeSignature \time 4/4 c'1 \time 2/2 c'1 }""",
'partial_start': r"""{ \partial 8 g'8 | c''4 d'' e'' f'' | g''1 }""",
'tempo': r"""{ \tempo "Allegro" 4 = 120 c'1 | \tempo 2 = 60 c'1 }""",
'tempo_text': r"""{ \tempo "Andante" c'1 }""",
'marks': r"""{ \mark \default c'1 | \mark \default c'1 | \mark #5 c'1 }""",
'ottava': r"""{ \ottava #1 c'''4 d''' e''' f''' | \ottava #0 c''1 }""",
'tremolo': r"""{ c'4:16 d'2:32 e'4 | \repeat tremolo 4 { c'16 e' } c'2 }""",
'glissando': r"""{ c'2\glissando g'2 }""",
'breaks': r"""{ c'1 \break d'1 \pageBreak e'1 }""",
'breathe': r"""{ c'2 \breathe d'2 }""",
'text_scripts': r"""{ c'4^"dolce" d'_"espr." e'^\markup { \italic rit. } f' }""",
'voices_separator': r"""\new Staff { << { c''4 d'' e'' f'' } \\ { a'2 g' } >> }""",
'explicit_voices': r"""\new Staff << \new Voice { \voiceOne c''1 } \new Voice { \voiceTwo a'1 } >>""",
'voice_names_lyrics': r"""\score { << \new Staff \new Voice = "mel" { c'4 d' e' f' } \new Lyrics \lyricsto "mel" { one two three four } >> }""",
'variables': r"""melody = \relative c' { c4 d e f } bass = { c1 } \score { << \new Staff \melody \new Staff { \clef bass \bass } >> }""",
'overrides': r"""{ \override NoteHead.color = #red c'4 \revert NoteHead.color d' \once \override Stem.transparent = ##t e' \tweak color #blue f' }""",
'set_unset': r"""{ \set Staff.instrumentName = "Vln" \unset Staff.instrumentName c'1 }""",
'stems': r"""{ \stemUp c'4 d' \stemDown e' f' \stemNeutral g'1 }""",
'accidentals': r"""{ cis'4 des' eisis' feses' | c'! c'? c'2 }""",
'long_notes': r"""{ c'\breve c'1 }""",
'dotted': r"""{ c'4. d'8 e'8. f'16 g'4 | a'2.. b'8 }""",
'bar_types': r"""{ c'1 \bar "||" c'1 \bar ".|:" c'1 \bar ":|." c'1 \bar "|." }""",
'figured_bass': r"""\score { << \new Staff { c'1 } \new FiguredBass \figuremode { <6 4>1 } >> }""",
'layout_midi': r"""\score { { c'1 } \layout { } \midi { } }""",
'book': r"""\book { \score { { c'1 } } }""",
'with_block': r"""\new Staff \with { instrumentName = "Flute" shortInstrumentName = "Fl." } { c''1 }""",
'transposed_instrument': r"""\new Staff { \transposition bes c''1 }""",
'empty_chord': r"""{ <>\p c'4 d' e' f' }""",
}

# known bugs outside the planned improvements
KNOWN_INVALID = {
'header': 'creator is written after encoding in identification',
'rhythmic_staff': 'isolated durations without a previous pitch crash (hidden)',
'drums': 'voice element comes after type in unpitched notes',
}


def corpus_params():
"""Return the corpus names as pytest parameters, known bugs marked xfail."""
return [pytest.param(name, marks=pytest.mark.xfail(strict=True, reason=KNOWN_INVALID[name]))
if name in KNOWN_INVALID else name
for name in sorted(CORPUS)]
106 changes: 106 additions & 0 deletions tests/musicxml_helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
"""Helpers for semantic tests of the LilyPond to MusicXML export.

The tests built on these helpers look at the parts of the MusicXML output they
are about (pitches, durations, barlines, ...) instead of comparing whole files.
"""
import os.path
from fractions import Fraction

from lxml import etree

import ly.musicxml


TESTS_DIR = os.path.dirname(__file__)
XSD_3_0 = os.path.join(TESTS_DIR, 'musicxml.xsd')
XSD_4_0 = os.path.join(TESTS_DIR, 'musicxml-4.0', 'musicxml.xsd')

_schemas = {}


def convert(ly_text):
"""Convert LilyPond text and return the root element of the MusicXML."""
writer = ly.musicxml.writer()
writer.parse_text(ly_text)
return etree.fromstring(writer.musicxml().tostring())


def validate(root, xsd=XSD_3_0):
"""Raise etree.DocumentInvalid if root is not valid against xsd."""
if xsd not in _schemas:
_schemas[xsd] = etree.XMLSchema(etree.parse(xsd))
_schemas[xsd].assertValid(root)


def parts(root):
return root.findall('part')


def measures(root, part=0):
return parts(root)[part].findall('measure')


def notes(root, part=0):
return parts(root)[part].findall('measure/note')


def pitch(note):
"""Return the pitch of a note element as e.g. 'C4', 'F#4' or 'Bb3'.

Rests are returned as 'r'.
"""
if note.find('rest') is not None:
return 'r'
p = note.find('pitch')
alter = {-2: 'bb', -1: 'b', 0: '', 1: '#', 2: '##'}[int(p.findtext('alter', '0'))]
return p.findtext('step') + alter + p.findtext('octave')


def pitches(root, part=0, voice=None):
"""Return the pitches of all notes in a part, optionally only of one voice."""
return [pitch(n) for n in notes(root, part)
if voice is None or n.findtext('voice') == str(voice)]


def divisions(root, part=0):
return int(parts(root)[part].findtext('measure/attributes/divisions'))


def quarters(root, part=0):
"""Return a function that converts a duration in divisions to quarters."""
div = divisions(root, part)
return lambda duration: Fraction(int(duration), div)


def voice_lengths(measure, div):
"""Return a dict mapping each voice to its length in quarter notes.

Chord notes and grace notes are not counted; <backup> and <forward>
move the current position, as in MusicXML.
"""
lengths = {}
pos = Fraction(0)
for el in measure:
if el.tag == 'note':
if el.find('chord') is not None or el.find('grace') is not None:
continue
pos += Fraction(int(el.findtext('duration')), div)
voice = el.findtext('voice', '1')
lengths[voice] = max(lengths.get(voice, 0), pos)
elif el.tag == 'backup':
pos -= Fraction(int(el.findtext('duration')), div)
elif el.tag == 'forward':
pos += Fraction(int(el.findtext('duration')), div)
voice = el.findtext('voice', '1')
lengths[voice] = max(lengths.get(voice, 0), pos)
return lengths


def measure_lengths(root, part=0):
"""Return the length of every measure in quarter notes.

The length of a measure is the length of its longest voice.
"""
div = divisions(root, part)
return [max(voice_lengths(m, div).values(), default=0)
for m in measures(root, part)]
Loading