SegmentComparer 2.2.0

SegmentComparer

Compare two versions of a text segment and see what was added, removed and left unchanged, by word or by character. Tags and placeholders are compared as whole units, which makes it a good fit for translation segments (XLIFF, Trados Studio) and for showing edits as track changes.

• Targets: .NET Framework 4.8 and .NET 8
• Dependencies: Newtonsoft.Json
• NuGet: https://www.nuget.org/packages/SegmentComparer

Each comparison returns:

• the comparison units: runs of text or tags marked Identical, New or Removed, in reading order;
• the edit distance (Damerau-Levenshtein, in characters, each tag counting as one) and the length it relates to;
• word, character and tag counts for both versions.

Install

• dotnet CLI: dotnet add package SegmentComparer

Quick start

using System;
using System.Linq;
using SegmentComparer;
using SegmentComparer.Structure;

var comparer = new Comparer();

var original = "The quick brown fox";
var updated  = "The quick red fox";

// Defaults: includeTags=true, tagVisualType=2 (FullTagText), semanticGrouping=true, comparisonType=0 (words)
var result = comparer.CompareSegment("seg-1", original, updated);

// Simple projection of results
var added   = result.ComparisonUnits.Where(u => u.Type == ComparisonUnit.ComparisonType.New).Select(u => u.Text);
var removed = result.ComparisonUnits.Where(u => u.Type == ComparisonUnit.ComparisonType.Removed).Select(u => u.Text);
var same    = result.ComparisonUnits.Where(u => u.Type == ComparisonUnit.ComparisonType.Identical).Select(u => u.Text);

Console.WriteLine("Added: " + string.Join(", ", added));
Console.WriteLine("Removed: " + string.Join(", ", removed));
Console.WriteLine("Unchanged: " + string.Join(", ", same));

// Edit distance
int relative;
int distance = comparer.GetSegmentEditDistance(original, updated, out relative, includeTags: true);
Console.WriteLine($"Edit distance: {distance} of {relative}");

Preview: rebuilt string with highlighting

Markdown-safe (renders everywhere; strike-through for removed, plain text for added/unchanged):

• The quick brown red fox

HTML-styled (for environments that allow inline styles; added=blue, removed=red strike, identical=black): <div> The quick <span style="color:#d32f2f;text-decoration:line-through;text-decoration-thickness:2px;text-decoration-color:#d32f2f;"> brown</span> <span style="color:#1976d2;"> red</span> fox </div>

Note: nuget.org sanitizes styles, so the HTML colors may not render there. The Markdown-safe example will still show the strike-through.

How to generate this output

using System;
using System.Linq;
using System.Net;
using System.Text;
using SegmentComparer;
using SegmentComparer.Structure;

// 1) Compare
var comparer = new Comparer();
var original = "The quick brown fox";
var updated  = "The quick red fox";
var result   = comparer.CompareSegment("seg-1", original, updated);

// 2) Build Markdown-safe text (strike for removed, plain for others)
static string BuildMarkdown(ComparisonResult result)
{
    var sb = new StringBuilder();
    foreach (var u in result.ComparisonUnits)
    {
        var text = u.Text; // already plain text at this point
        switch (u.Type)
        {
            case ComparisonUnit.ComparisonType.Removed:
                sb.Append("~~").Append(text).Append("~~");
                break;
            default:
                sb.Append(text);
                break;
        }
    }
    return sb.ToString();
}

// 3) Build HTML with colors (blue added, red strike removed, black identical)
static string BuildHtmlInline(ComparisonResult result)
{
    string StyleFor(ComparisonUnit.ComparisonType t) => t switch
    {
        ComparisonUnit.ComparisonType.New       => "color:#1976d2;",
        ComparisonUnit.ComparisonType.Removed   => "color:#d32f2f;text-decoration:line-through;text-decoration-thickness:2px;text-decoration-color:#d32f2f;",
        _                                        => "color:#000;"
    };

    var sb = new StringBuilder();
    sb.Append("<div class=\"diff\">");
    foreach (var u in result.ComparisonUnits)
    {
        var text = WebUtility.HtmlEncode(u.Text);
        sb.Append("<span style=\"").Append(StyleFor(u.Type)).Append("\">")
          .Append(text)
          .Append("</span>");
    }
    sb.Append("</div>");
    return sb.ToString();
}

// Examples
var md   = BuildMarkdown(result);
var html = BuildHtmlInline(result);

Console.WriteLine("Markdown output:");
Console.WriteLine(md);

Console.WriteLine("\nHTML output:");
Console.WriteLine(html);

If you need character-level comparison, call CompareSegment with comparisonType: 1.

Comparing text and tags as sections

The string overload treats anything in angle brackets as a tag. When your text can contain < and > that are not tags, or you already have the segment as text and tags (for example from an editor), pass sections instead. Your sections are not changed by the comparison.

var original = new List<Section>
{
    new Text { Value = "Press " },
    new Tag { Id = "1", Text = "<b>", Type = Tag.TagType.Opening },
    new Text { Value = "Start" },
    new Tag { Id = "1", Text = "</b>", Type = Tag.TagType.Closing }
};
var updated = new List<Section>
{
    new Text { Value = "Press " },
    new Tag { Id = "1", Text = "<b>", Type = Tag.TagType.Opening },
    new Text { Value = "Begin" },
    new Tag { Id = "1", Text = "</b>", Type = Tag.TagType.Closing }
};

var result = new Comparer().CompareSegment("seg-2", original, updated);

Notes

• Whitespace: a space after the first in a run of whitespace is returned as a no-break space (U+00A0), so it stays visible when rendered. Tabs and line breaks are returned as they are.
• Threads: Comparer holds no state; one instance can be used from several threads.
• Null input is treated as an empty segment.

What's new in 2.2.0

• Targets .NET Framework 4.8 again, alongside .NET 8.
• Fixed: comparing sections no longer changes the caller's sections (adjacent text sections were joined into the first one, which also made the word counts and edit distance wrong).
• Fixed: comparisons running at the same time with different settings could use each other's settings.
• Fixed: a changed tab or line break is detected, and whitespace keeps its order when changes are grouped.
• Fixed: emoji and other surrogate pairs are compared and returned whole.
• Fixed: tag ids are kept, also when they contain quotes; a tag without text and null text no longer throw.
• Revision and Section.Revision are obsolete; remove deleted sections before comparing instead.

Attribution

AB testing icons created by Bharat Icons — Flaticon