FactMiners.org - Neo4j

Congratulations CIDOC-CRM SIG and the Year Ahead for #cidocCRM Development

Jim Salmons — Tue, 03 Feb 2015 20:08:57 +0000

FactMiners joins the Library, Archive, and Museum (LAM) tech community in congratulating the members of the CIDOC-CRM Special Interest Group for its kick-off to 2015 with release of Version 6.0 of the International Council of Museums (ICOM) Conceptual Reference Model. In addition to the major point update to the model's Reference Document, the CIDOC-CRM SIG also announced a collaboration with the UK-based Collections Trust to create a CIDOC-CRM extension for the Trust's widely-used and highly-regarded SPECTRUM standard for museums' collections management systems.

The CIDOC-CRM (tweeted/hashtagged here and elsewhere as #cidocCRM) serves as a metamodel providing definitions and a formal structure for describing the implicit and explicit concepts and relationships used in cultural heritage documentation. FactMiners is committed to #cidocCRM-driven design for our Open Source social game and exploratory personal and professional "serious fun" research platform.

"Early adopter use cases in the museum community are understandably focused on the descriptive, or ontological, aspects of the CIDOC-CRM. Designing and developing new software systems based on a rigorous metamodel is not the first thing you do with such a resource." said Jim Salmons, tech lead, researcher, and founder of FactMiners and The Softalk Apple Project, "We are very encouraged by publication of the new edition of the Reference Document together with the SPECTRUM announcement. Given the mission and community of The Collections Trust, this is very good news for any museum techies and researchers interested in #cidocCRM-based development projects."

"For our part," Salmons continued, "FactMiners is announcing the launch of our 'Year of the #cidocCRM Full-Graph Deep Dive'. We're asking Kindred Spirits interested in #cidocCRM-driven software design and development to tweet and tag their posts and communications with #cidocCRMdev and #cidocCRMgraph. These hashtags will help focus communication within the museum and broader LAM tech communities specific to these important emerging topics."

In addition to rallying interest in #cidocCRM-driven development within the museum informatics community, FactMiners will increase its outreach activity to engage the Neo4j and broader graph database communities in the exploration and development of a "full graph" implementation of the #cidocCRM.

FactMiners will also engage Neo Technology, the Structr folks, and vendors in the graph visualization community, like KeyLines, to join us in making a contribution to the global cultural preservation movement by working together this year to create a new Neo4j/Structr-based replacement for the www.CIDOC-CRM.org community website. This site is used as a central repository for the #cidocCRM model and its various information resources and news about this ISO standard metamodel. As new developers embrace the #cidocCRM, the community website will be increasingly important. An interactive Neo4j and Structr based site with "live document" visualizations and cross-referencing for the #cidocCRM metamodel will be an ideal way to showcase our web, app, and visualization technologies while making an important contribution to global cultural preservation.

"2015 looks to be the year that the CIDOC-CRM gets a real 'shakedown cruise' as a broader community of software designers and developers begin to explore and use this valuable resource within the cultural heritage community." Salmons said, "And FactMiners is happy to be a part of these exciting developments in the evolution of Internet and its use in the preservation and transmission of our cultural heritage. Keep up the great work CIDOC-CRM SIG! Full speed ahead #cidocCRMdev and #cidocCRMgraph!"

Thoughts on a Graph Representation of the CIDOC-CRM: Property Declarations

Jim Salmons — Sun, 02 Nov 2014 00:12:09 +0000

I am having an interesting time looking at the CIDOC-CRM – the Conceptual Reference Model for museums developed by the International Council of Museums (ICOM). In particular, as generally described in my last post, I am looking at the CIDOC-CRM with a "pure graph" lens on a metamodel developed with an object-oriented perspective.

The interesting part of my exploration has to do with bringing the "ruthless simplicity" of a graph expression to the interpretation of the CIDOC-CRM for the purpose of representing it as a Referenced Model within the metamodel partition of the graph database as part of the FactMiners' Digital Humanities-based social-game platform. I've written two complementary Neo4j-based GraphGists that explore the design pattern of an "embedded metamodel subgraph" as a means to build a fine-grained "Fact Cloud" of the referential content of the 48-issue collection of Softalk magazine covering the early history of the microcomputing revolution.

In this post, I'll take a first look at how CIDOC-CRM Property Declarations can be beneficially cast as "First Class Citizens" (AKA Nodes) in a graph model of the CIDOC-CRM. I am still in the very early stages of CIDOC-CRM exploration, so take these observations as public research notes.

The Challenges and Potential of the CIDOC-CRM

An interesting dimension of the CIDOC-CRM is that it intentionally tries to be a comprehensive conceptual model (AKA a metamodel) for not just museum artifact description, but to cover the active process of artifact curation and preservation as well as broader realms of collection management, etc. In this sense, the CIDOC-CRM goes where most cultural artifact ontologies rarely go, that is, into the realm of incorporating process along with structure-oriented model elements from which to construct our CRM-compatible model instances.

That's what a Conceptual Reference Model or metamodel is used for, at its most basic level. It's a set of model elements (parts to build a model out of) and a set of instructions that constrain how you can put these parts together. Through this model, we can share an understanding about what some domain of activity and expertise is about. That understanding is to be used, in our case, to design software systems to implement this shared conceptual (mental) understanding of what needs doing and how to do it. (I know this is not the only reason for having a comprehensive conceptual reference or metamodel, but it is the compelling reason for this exploration and discussion.)

As is often the case, industry reference models are rarely used to drive down to the granularity of "executable" in the sense of software being designed strictly to the, let's call it "semantic expressibility" of the metamodel. In other words, these conceptual reference models are most often used loosely to guide human-to-human ("mind-meld") conversation and modeling-based decision-making. But there is a lot of room for what are in effect "uncontrolled individual model transformations" (AKA we developers think about how to implement the model and then write some code that approximates our personal understanding of this shared vision). So if during implementation of metamodel-driven design there are "gotchas" or logical inconsistencies or just plain "temporal anomalies" (a big model done by committee over a long period of time), these issues are not too often show-stopping problems as we "one off" resolve these issues by "coding band-aids" as we go along.

But one of the goals that the CIDOC-CRM SIG group members aspire to encourage and support is research-based inference and deductibility – text- and image-based computational analytics – for CIDOC-CRM compatible datastores and associated #LODLAM web services. In other words, "close enough for horseshoes or hand grenades" may be good enough for many cases of using a conceptual reference model for indirect reference, but this is not good enough if we are going for fact- or inference-discovery and validation via metamodel-constrained computational analytics.

So let's look at the most obvious case in point; Property Declarations as "faux" relationships between CRM Classes.

CIDOC-CRM Property Declarations as "Shortcut" Classes

A typical CIDOC-CRM graphical diagram of key parts of the model shows the graph-based shared understanding of how we're "thinking about" this museum conceptual model. The "object"-like things are drawn as nodes and have "E-prefix" names that reflect their "IS_A" (object-oriented inheritance) descent from the "E1 CRM Entity" root of the CIDOC-CRM Class Hierarchy. The relationships (or edges in a pure graph sense) between Classes (AKA our graph-based Nodes) are defined as CIDOC-CRM Properties. CIDOC-CRM Properties have a "Pnum-prefix" name and have the additional facet of "bidirectional/reflexive naming" -- e.g. P67 "refers to/is referred to by" providing the appropriate human-readable interpretation of the named relationship based on which "end" of the relationship you want to anchor your mental understanding of the described relationship.

To understand how this current view on the CIDOC-CRM is problematic to a "pure graph" expression, let's look at a particularly interesting cluster of CIDOC-CRM model elements (both Classes and Properties) that describe the "Image Information, Objects and Carriers". I've highlighted the aspects of this diagram of the CIDOC-CRM definition that cannot be expressed in a basic graph representation.

This subset of CIDOC-CRM model elements is especially interesting as it gets to the core of our FactMiners' design interest – and that is, "piercing the veil" to move our Fact Cloud coverage from artifact description to deep analysis by modeling the artifact's representational meaning. In our case, we require a metamodel sufficient to cover the complex structure of a commerical magazine while providing the granularity to model – based on CIDOC-CRM-compatible form – the 48 monthly snapshots of the cacophonous "Open World" of activity at the dawn of the Microcomputer Revolution as depicted in the incredible content of this historic magazine archive.

The eagle-eyed "graphistas" among you will notice a show-stopping gotcha. Those double-line arrows from relationship-to-relationship violate the essential "ruthless simplicity" of a pure graph model. A graph is just nodes and relationships, with relationships being the edge/line links BETWEEN nodes. There are no relationships between relationships in basic graph theory.

Fortunately, this is not a show-stopping problem. It just suggests that the view we're given is more a "mental shortcut" for a graph expression that models the relationship as a distinct type of Class. In this way, a "node-ified" CIDOC-CRM Property can be modeled with respect to its "IS_A" subproperty/superproperty relationships as well as support the "Pnum.1" idiom where property specializations can be expressed through "mode of X" relationships to "E55 Type" entities, etc.

In other words, here is how we can look at a portion of the "Image Information, Objects and Carriers" model with "node-ified" CIDOC-CRM Property Declarations:

In this diagram, the "subset-membership containment areas" are a visualization of, in Neo4j graph database-terms, a label-based conceptual type. That is, our metamodel of the CIDOC-CRM contains Nodes that are of type "CRM Class" and a collection of Nodes of type "CRM Property." These two labeled subsets are emphasized in the graphic by node color in addition to their containment in subset-membership labeled boxes.

Now that we have a "node-ified" Property "P138 represent/has representation" represented in our graph as a node, we can create its "subproperty of" relationship to "P76 refers to/is referred to by." This "subproperty of" relationship, by the way, can be thought of as a specialization of the "IS_A" relationship underlying the object-oriented CIDOC-CRM Class subclass and superclass inheritance relationships. Note, too, that we can easily create the "Pnum.1" relationships that provide specialization of a CRM Property.

[Note: The above diagram is decidedly high-level. A full rendering of a "pure graph" expression of the CIDOC-CRM would show labeled domain/range relationships, cardinality, etc. These fine-grained aspects of the model are left off here to keep the explanatory diagram cleaner.]

Okay, but so what? Why "node-ify" CRM Properties?

At some level we could think, "Okay, so what?" Aren't we just nit-picking over the details of the metamodel representation for communication and system design discussions. For many, yes, that's true. But the FactMiners LAM-based social-game platform is being designed with an underlying cognitive computing perspective. We need our metamodel expression of the CIDOC-CRM to be machine-executable.

This aspiration for deep computational analysis of cultural artifacts is also one held by members of the CIDOC-CRM Special Interest Group. Dominic Oldman – a CIDOC-CRM SIG member, Principle Investigator of www.ResearchSpace.org, and IT exec of The British Museum – writes in the latest version of "The CIDOC Conceptual Reference Model (CIDOC CRM): PRIMER" (my emphasis added):

"The most important kinds of computer-based reasoning the CRM can support are generalisations of relationships and deductions from highly indirect relations such as what parts have in common with their wholes, what wholes inherit from their parts and what is transferred across meetings and processes of derivation. These are not meant to replace scholarly conclusions but to comprehensively detect facts relevant to answer research questions. Besides others this ensures that highly specialized knowledge stays accessible to generic questions regardless the specificity of representation."

My belief is that a fully-realized graph expression of the CIDOC-CRM will not just support the kind of "fact-checking" and discovery that Dominic asserts in the quote above, but will move CIDOC-CRM-based computational analytics into the realm of significant first-class, fine-grained, traceable, interpretive scholarship that is only just beginning to be imagined based on the rapid advances in Deep Learning, Natural Language Processing, Image Scene Recognition, and other aspects of the explosion of advances in what is being loosely corralled by the term "Cognitive Computing."

While my example is ruthlessly simple for the purpose of introductory explanation and is not CIDOC-CRM-specific, the first part of my two-part GraphGist demonstrates how generalized query-based fact discovery and validation can be performed on a "self-descriptive" Neo4j-based graph database. And closely related to this GraphGist is the research agenda described in my "Rainman Meet Sherlock" post.

I welcome your thoughts and/or questions.

Wishlist: A Metamodel Partition in Neo4j Graph Database

Jim Salmons — Sun, 30 Mar 2014 01:17:50 +0000

There are many reasons I had to explore the embedded metamodel subgraph design pattern for Neo4j databases in the first series of FactMiners ecosystem design documents.

My interest is grounded in my experience in the 1990's developing a pair of complementary Distributed Smalltalk frameworks to do what we called "executable business models." The basic idea was that if we came up with a super-elegant metamodel about how to do business processes AND STUCK TO IT no matter what on the server/executing-model side of things then the Desktop Visualization framework could dynamically generate what customer/users mistook for "applications."

At the time, this required us to do something that was considered heretical in the OOP community which was to explicitly objectify BusinessProcess and similar "non-object" objects. While the OOP purists poo-pooed what we were doing, we found INCREDIBLE leverage in design-to-implementation and stakeholder buy-in (as what we built made sense to them... it was their mental model embodied in software).

We can bring this kind of leverage to graph databases, and I hope Neo4j will be the pioneer leader in this, by coming up with a community standard for an embedded metamodel subgraph feature. By adopting the general embedded metamodel subgraph pattern and defining a core structure for the general semantics of a metamodel in a graph database, we would then have a common mechanism for 3rd party product/service developers to hang their tool-specific "decorations"/hints/config-specs/whatever. With some kind of lightweight mechanism for registering a property namespace in the META partition (or whatever this subgraph is called), 3rd party devs would be able to work together more efficiently to provide interoperability and other pipeline-type features that are all good for the Neo4j ecosystem.

While nothing technically would need to be done in Neo4j core to create and use such a design pattern, there would be a couple helpful things:

As part of its "schema-like support" Neo4j could support an OPTIONAL configuration directive to name the label of an embedded non-connected subgraph, e.g. META.
While our interest is in using this for an embedded metamodel, there doesn't need to be any requirement on its use other than this is a "database within a database" such that it can be systematically IGNORED in Cypher queries unless explicitly referenced. (Not sure of the technical feasibility of this systematic exclusion. But even without it, I believe the pattern can be used without too much chance of 'result contamination' providing best practices for naming conventions are followed, etc.)
While not strictly required and not limited in usefulness to metamodel modeling, I would like to see some kind of optional PATH SEMANTIC for LABELS to express subset containment and not just subset membership. E.g., (foo:Man.Chu) is different than (foo:Man:Chu) in that in the first, Chu is a subset of Man.

Subset containment would be EXTREMELY helpful for organizing a metamodel as explored in our first FactMiners GraphGists. The best solution for adding a label path semantic is one that would provide a pattern-matching syntax in Cypher so users would not need to resort to regexes, etc. when doing metamodel discovery, interpretation, and application or visualization configuration, etc.

I certainly see where this design pattern could be used to provide a "freeze-dry" mechanism for in-graphdb storage and transmission of Structr-specific info. I would love to see it where you load a new Neo4j database into Structr, it finds a META partition and grabs all the Structr-specific info the database creator/updater provides. If Structr, upon examining the metamodel finds KeyLines "decorations", for example, and the current Structr user has KeyLines installed, Structr/CambridgeIntelligence can work out what that combination means and provide tool-2-tool configuration... again, to be stored in the shared embedded metamodel once that collaboration is determined.

I am looking at this strictly from the Wish List of an "itch scratching" developer. I do not know the particulars of what it would take to implement such an optional ignorable subgraph partition. I just know what I'd do with one if we had it. :-)

Thoughts?