尽可能高效地编写和流式传输模板化 Xml

Question

我正在尝试寻找最佳解决方案，以防止我们在构建 Xml 文档时分配过多内存。我必须用尽可能少的资源构建一个相当大的 Xml（Web 服务必须能够每秒处理数百个调用）。 Xml 本身的结构没有太大变化，但是数据一直在变化。 我目前的解决方案是 XDocument 和 XElement (LINQ)。下面是我今天正在做的一个简单示例：

static Stream GetXml(string d1, string d2, string d3)
{
    XElement x = 
        new XElement("myElement",
            new XElement("myOtherElement1", d1),
            new XElement("myOtherElement2", d2),
            new XElement("myOtherElement3", d3));
    // ... more XElement
    // ... return Stream
}

当 Xml 文档变得太大时，实例化一个 XDocument 和数百个 XElement 会变得非常昂贵，并且每秒的调用次数会下降。 我目前正在考虑创建某种模板引擎，它可以简单地流式传输字符串（XElement）而不实例化任何对象。你会怎么做？这是正确的做法吗？

static Stream GetXml(string d1, string d2, string d3)
{
    const string xml = @"
<myElement>
  <myOtherElement1>{0}</myOtherElement1>
  <myOtherElement2>{1}</myOtherElement2>
  <myOtherElement3>{2}</myOtherElement3>
</myElement>";

    // What's the best way to {0}, {1}, {2} without allocating 
    // objects and using as little RAM as possible. I cannot 
    // use string.Format since it allocates strings.

    StreamWriter sw = new StreamWriter(stream);
    sw.Write(xml);
}

Answer 1

如果您想避免使用正在解析/生成的 XML 加载内存，请考虑使用在 http://msdn.microsoft.com/en-us/library/system.xml.linq.xstreamingelement.aspx 中讨论的 XStreamingElement 实现。

Answer 2

我能想到的解雇string.Format 的唯一原因是您不想一次将整个XML 文档保存在内存中。我编写了这个 Stream 类，当时应该只在内存中保留一小部分文档。

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;

namespace StreamTest
{
    public class EnumeratorStream : Stream
    {
        private readonly IEnumerator<string> source;
        private readonly Encoding encoding;

        public Encoding Encoding { get { return encoding; } }

        private byte[] current = new byte[0];
        private int currentPos = 0;

        public EnumeratorStream(IEnumerable<string> source, Encoding encoding)
        {
            if (source == null) throw new ArgumentNullException("source");
            if (encoding == null) encoding = Encoding.Default;

            this.source = source.GetEnumerator();
            this.encoding = encoding;
        }

        private bool MoveNext()
        {
            while (source.MoveNext())
            {
                if (source.Current.Length > 0)
                {
                    current = encoding.GetBytes(source.Current);
                    currentPos = 0;
                    return true;
                }
            }
            current = new byte[0];
            currentPos = 0;
            return false;
        }

        #region Overrides of Stream

        public override bool CanRead { get { return true; } }
        public override bool CanSeek { get { return false; } }
        public override bool CanWrite { get { return false; } }

        public override int Read(byte[] buffer, int offset, int count)
        {
            if (buffer == null) throw new ArgumentNullException("buffer");
            if (offset < 0) throw new ArgumentOutOfRangeException("offset");
            if (offset + count > buffer.Length) throw new ArgumentException("Not enough buffer space");

            int totalWritten = 0;
            while (count > 0)
            {
                int remaining = current.Length - currentPos;
                if (remaining == 0 && !MoveNext()) break;
                remaining = current.Length - currentPos;
                if (remaining <= 0) break;
                if (remaining > count)
                {
                    remaining = count;
                }
                Array.Copy(current, currentPos, buffer, offset, remaining);
                offset += remaining;
                count -= remaining;
                totalWritten += remaining;
                currentPos += remaining;
            }
            return totalWritten;
        }

        public override void Flush() { }
        public override long Seek(long offset, SeekOrigin origin) { throw new NotSupportedException(); }
        public override void SetLength(long value) { throw new NotSupportedException(); }
        public override void Write(byte[] buffer, int offset, int count) { throw new NotSupportedException(); }
        public override long Length { get { throw new NotSupportedException(); } }
        public override long Position
        {
            get { throw new NotSupportedException(); }
            set { throw new NotSupportedException(); }
        }

        #endregion
    }
}

例子：

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;

namespace StreamTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var stream = new EnumeratorStream(Generate("x","y","z"), null);
            var buffer = new byte[256];
            int read;
            while ((read = stream.Read(buffer,0,256)) > 0)
            {
                string s = stream.Encoding.GetString(buffer, 0, read);
                Console.Write(s);
            }
            Console.ReadLine();
        }

        public static IEnumerable<string> Generate(string d1, string d2, string d3)
        {
            yield return "<myElement>";
            yield return "<myOtherElement1>";
            yield return d1;
            yield return "</myOtherElement1>";
            yield return "<myOtherElement2>";
            yield return d2;
            yield return "</myOtherElement2>";
            yield return "<myOtherElement3>";
            yield return d3;
            yield return "</myOtherElement3>";
            yield return "</myElement>";
        }
    }
}

Answer 3

您可以传递一个 StringBuilder。任何重复的字符串（如开始和结束标签）都会引用内存中的相同数据，因此您可以节省一些费用。

static Stream GetXml(string d1, string d2, string d3)
{
    StringBuilder xml = new StringBuilder();
    xml.Append("<myElement>");
    AppendElement(xml, d1);
    AppendElement(xml, d2);
    AppendElement(xml, d3);
    xml.Append("</myElement>");

    // Create/return stream
}
static void AppendElement(StringBuilder xml, string value)
{
    xml.Append("<myOtherElement>");
    xml.Append(value);
    xml.Append("</myOtherElement>");
}

为了节省更多，您可以像这样拼凑开始和结束元素：

static Stream GetXml(string d1, string d2, string d3)
{
    StringBuilder xml = new StringBuilder();
    OpenElement(xml, "myElement");
    AppendElement(xml, d1);
    AppendElement(xml, d2);
    AppendElement(xml, d3);
    CloseElement(xml, "myElement");

    // Create/return stream
}
static void AppendElement(StringBuilder xml, string value)
{
    OpenElement(xml, "myOtherElement");
    xml.Append(value);
    CloseElement(xml, "myOtherElement");
}    

static void OpenElement(StringBuilder xml, string elementName)
{
    xml.Append("<");
    xml.Append(elementName);
    xml.Append(">");
}
static void CloseElement(StringBuilder xml, string elementName)
{
    xml.Append("</");
    xml.Append(elementName);
    xml.Append(">");
}