COM Structured Storage

<h2 id="motivation">Motivation</h2>
<p>Structured storage addresses some inherent difficulties of storing multiple data objects within a single file. One difficulty arises when an object persisted in the file changes in size due to an update. If the application that is reading/writing the file expects the objects in the file to remain in a certain order, everything following that object's representation in the file may need to be shifted backward to make room if the object grows, or forward to fill in the space left over if the object shrinks. If the file is large, this could result in a costly operation. Of course, there are many possible solutions to this difficulty, but often the application programmer does not want to deal with low level details such as binary file formats.
</p><p>Structured storage provides an abstraction known as a <i>stream</i>, represented by the interface IStream. A stream is conceptually very similar to a file, and the IStream interface provides methods for reading and writing similar to file input/output. A stream could reside in <a href="/facts/Random-access_memory/DuD4bxh6">memory</a>, within a file, within another stream, etc., depending on the implementation. Another important abstraction is that of a <i>storage</i>, represented by the interface IStorage. A storage is conceptually very similar to a <a href="/facts/Directory_(file_systems)/VLtLcEXY">directory</a> on a <a href="/facts/File_system/FdPm90uc">file system</a>. Storages can contain streams, as well as other storages.
</p><p>If an application wishes to persist several data objects to a file, one way to do so would be to open an IStorage that represents the contents of that file and save each of the objects within a single IStream. One way to accomplish the latter is through the standard COM interface IPersistStream.  OLE depends heavily on this model to embed objects within documents.
</p>
<h2 id="format">Format</h2>
<p class="note">Main article: <a href="/facts/Compound_File_Binary_Format/7D9qushB">Compound File Binary Format</a></p>
<p>Microsoft's implementation uses a file format known as <i>compound files</i>, and all of the widely deployed structured storage implementations read and write this format.  Compound files use a <a href="/facts/File_allocation_table/uFvpmWxa">FAT</a>-like structure to represent storages and streams. Chunks of the file, known as <i>sectors</i> (these may or may not correspond to sectors of the underlying file system), are allocated as needed to add new streams and to increase the size of existing streams. If streams are deleted or shrink, leaving unallocated sectors, those sectors can be reused for new streams.
</p><p>The following applications use the OLE Structured Storage (Compound Document Format)
</p>
<ul><li><a href="/facts/Microsoft_Office/faD8vYG0">Microsoft Office</a> <a href="/facts/Office_97/WcXLL1Jy">97</a>–<a href="/facts/Office_2003/wnwT6suV">2003</a> documents:
<ul><li><a href="/facts/Microsoft_Word/HPSu5W4j">Word</a> documents (.DOC, .DOT)</li>
<li><a href="/facts/Microsoft_Excel/Ezal6G75">Excel</a> spreadsheets (.XLS, .XLT)</li>
<li><a href="/facts/Microsoft_PowerPoint/putuX423">PowerPoint</a> presentations (.PPT, .POT)</li>
<li><a href="/facts/Microsoft_Publisher/J5AompnL">Publisher</a> files (.PUB)</li>
<li><a href="/facts/Microsoft_Visio/xvyc8v0L">Visio</a> files (.VSD)</li>
<li><a href="/facts/Microsoft_Project/ooboaNXh">Project</a> files (.MPP)</li>
<li><a href="/facts/Microsoft_PhotoDraw/raKIrDr2">Microsoft PhotoDraw</a> files (.MIX)</li>
<li><a href="/facts/Microsoft_Outlook/cMGCKGFJ">Microsoft Outlook</a> files (.MSG)</li></ul></li>
<li><a href="/facts/Windows_Installer/PGsKkYBr">Windows Installer</a> files (.MSI, .MSP, .MST)</li>
<li><a href="/facts/Microsoft_Picture_It!/FkbkJUY2">Microsoft Picture It!</a> / <a href="/facts/Microsoft_Digital_Image/VDswFoxs">Microsoft Digital Image</a> files (.MIX)</li>
<li>Internet Explorer RSS Feeds Windows RSS Platform files (.feed-ms)</li>
<li>Windows 7 StickyNotes (.SNT)</li>
<li>Windows 7 jumplists files</li>
<li><a href="/facts/Windows_thumbnail_cache/do0wasn9">Thumbs.db</a></li>
<li>Microsoft SQL 2000 Server <a href="/facts/Data_Transformation_Services/LEXCquAh">DTS</a> packages</li>
<li>Autodesk Revit</li>
<li>Autodesk Inventor</li>
<li><a href="/facts/FlashPix/JPNjfbFN">FlashPix</a></li>
<li><a href="/facts/Altium_Designer/FuM4lTkz">Altium Designer</a></li></ul>
<h2 id="native-structured-storage">Native Structured Storage</h2>
<p>During the <a href="/facts/Beta_testing/VoXzG9Tb">beta testing</a> phase of <a href="/facts/Windows_2000/amJIt9gz">Windows 2000</a>, it included a feature titled Native Structured Storage (NSS) for storage of Structured Storage documents (like the binary <a href="/facts/Microsoft_Office/faD8vYG0">Microsoft Office</a> formats and the thumbs.db file <a href="/facts/Windows_Explorer/oQW5yNnJ">Windows Explorer</a> uses to cache thumbnails) with each <i>Stream</i> that makes up a document stored in a separate <a href="/facts/NTFS/dryNM3BK">NTFS</a> <a href="/facts/Alternate_Data_Stream/dryNM3BK">data stream</a>. It included utilities that automatically split up the streams in a regular Structured Storage document into NTFS data streams and vice versa. However, the feature was withdrawn after Beta 3 due to incompatibilities with other OS components, and any NSS files automatically converted to the single data stream format.<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a>
</p>
<h2 id="implementations">Implementations</h2>
<ul><li>For Microsoft .NET:
<ul><li><a href="https://github.com/ironfede/openmcdf">OpenMCDF</a> – Free .NET component for accessing OLE structured storage files, <a href="/facts/Mozilla_Public_License/hBuvAz1w">MPL licensed</a>.</li></ul></li>
<li>For Linux:
<ul><li><a href="http://www.gnome.org/projects/libgsf/">GNOME Structured File Library</a> – Can read Microsoft structured storage files.</li>
<li><a href="https://web.archive.org/web/20130429210533/http://pole.berlios.de/">POLE</a>.</li></ul></li>
<li>Cross platform C++ for Window/MacOSX/Linux:
<ul><li><a href="http://www.dimin.net/software/pole/">POLE v3 and up</a>.</li></ul></li>
<li>For Java:
<ul><li><a href="https://web.archive.org/web/20070521192832/http://jakarta.apache.org/poi/poifs/index.html">POIFS</a> – Java implementation of the OLE 2 Compound Document format, part of <a href="/facts/Apache_POI/wv7Qvscv">Apache POI</a>.</li></ul></li>
<li>For Perl:
<ul><li><a href="https://web.archive.org/web/20091016040350/http://user.cs.tu-berlin.de/~schwartz/pmh/index.html">LAOLA Binary Structures</a></li></ul></li>
<li>For JavaScript:
<ul><li><a href="https://github.com/SheetJS/js-cfb">js-cfb</a> – JavaScript implementation of the OLE 2 Compound Document format.</li></ul></li>
<li>For Python:
<ul><li><a href="https://github.com/waveform-computing/compoundfiles">compoundfiles</a> – Python implementation of the Microsoft Compound File Binary (CFB) format.</li></ul></li></ul>

<h2 id="external-links">External links</h2>
<ul><li><a href="http://msdn.microsoft.com/en-us/library/aa380369%28VS.85%29.aspx">Official website</a></li>
<li><a href="http://msdn.microsoft.com/en-us/library/dd942138.aspx">Open Specifications: Compound File Binary File Format</a></li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>"What is Native Structured Storage?". Archived from the original on 2007-09-27. Retrieved 2007-12-03. <a href="https://web.archive.org/web/20070927212324/http://www.windowsitpro.com/Article/ArticleID/13785/13785.html" target="_blank">https://web.archive.org/web/20070927212324/http://www.windowsitpro.com/Article/ArticleID/13785/13785.html</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
</ol>

COM Structured Storage open-in-new

COM Structured Storage