FactMiners.org blogs

PRImA's Aletheia - Ground Truth & Softalk Magazine

Jim Salmons — Tue, 07 Jul 2015 22:51:27 +0000

Although most folks are now happy with full access to the complete run of the Apple edition of Softalk magazine now that Timlynn and I have funded its "ingestion" into the Internet Archive, the Softalk Apple Project is far from over. In fact, getting the collection into the Archive was just step one -- the Citizen History aspect of our projects.

We're now moving forward on strategic collaborations with world-class researchers and research centers to turn the Softalk Apple Project digital collection into a unique and valuable reference resource for broad applications in the Digital Humanities and Cognitive Computing domains.

One particularly exciting collaboration is shaping up with the VERY "deep weeds" researchers at the PRImA Research Center at the University of Salford in Manchester England. PRImA is "Pattern Recognition and Image Analysis" and this is the premiere research center working on the deepest technical challenges of doing document structure and layout recognition as well as OCR (text recognition) of print and hand-written documents.

PRImA's Aletheia - Our "Get-going" Fact-mining Tool

To date, the Center's research has focused (and made AMAZING progress) addressing within-page layout recognition via fine-grained page segmenting techniques. In order to test recognition algorithms, PRImA has created a "ground truth" tool called Aletheia. For the hard-core OCR crowd, "ground truth" is page segmentation meticulously done by, and validated to be "correct" as a perfect solution by, a human. The "ground-truth edition" of a page is used as an "answer key" to measure the accuracy of, and the differences between, the results of layout and text recognition algorithms.

The great thing is that FactMiners can use Aletheia as a "proof of concept" tool to begin creating the FactMiners' Fact Cloud "edition" of the Softalk Apple collection! A FactMiners' Fact Cloud is just our name for a "_machine-readable_ copy" of each issue of Softalk... that is, all the "facts mined" and stored in a graph database such that ALL the information that a human can gain by reading the magazine will also be accessible as #SmartData (that is, a graph database with a metamodel subgraph explaining the data's structure and processes for access and use) via the Fact Cloud.

Toward Automatic Recognition of Magazine Whole-Issue Structure

The picture I have included with this post is an example of detailed page-segmentation of the key "structure revealing" page of Softalk's first issue (September 1980). Page 1 is "hint rich" with information revealing the "meta-structure" (magazine's have a common but not required structure) of this issue of the magazine. In this case, page one has the Table of Contents, the Advertiser's Index (a 'secondary' type of TOC), the masthead, and Previews of next month's content.

Aletheia provides flexible region (page segment) creation, including the all-important page-segment-respecting OCR feature. Bulk OCR simply produces an unstructured "text soup" in an unseen layer of, for example, an image-based PDF file. While full-text searches can be done on such bulk OCR data, the actual and all-important structure of the magazine is nowhere to be found. This is a central challenge of FactMiners' fact-mining -- typed-structure-respecting text recognition... our Holy Grail of tool requirement.

The only way to create a FactMiners' Fact Cloud will be through a page-segment modeling and respecting tool... and to date, nothing I have seen comes close to filling this requirement the way Aletheia does.

FactMiners' "Visual Language of Magazine Design" Dataset

One of the #CognitiveComputing agenda items we're working on at FactMiners is a whole-issue commercial magazine layout recognition. This will be a specialized #SmartProgram that, given a set of images of the pages of a magazine, finds the page or pages that contain the "telltale specification" of the magazine -- that is, that finds and processes the table of contents, list of advertisers, and any other page-segments that reveal the whole-issue structure of the magazine. Finding and interpreting whole-issue magazine structure is a recognition process that spans individual pages and will be done in an iterative fashion over the collection of all page-images in an issue.

The whole-issue structure-recognizing challenge will require a "Sudoku-like" iterative solution. That is, by finding the "key page(s)" -- primarily the table of contents and list of advertisers -- our recognition algorithm or neural net will use a process of elimination to find and identify as many page segments as possible. While doing this at the page level, the #SmartProgram will be building up a whole-issue document structure as its iterative recognition reveals the front-of-book, feature-well, and back-of-book structure of most commercial magazines.

To this end, we'll be creating a computer-vision reference dataset of the "visual language" of magazine layout design. As we create the FactMiners' Fact Cloud, an intermediary step will be to create PAGE-format files (PRImA's XML spec similar to ALTO) that include a "whole page layout" description at the page level of the XML-based PAGE document structure hierarchy. This means that the Softalk Apple collection will be the first 9,300+ images of a dataset that can be used to teach deep learning algorithms and neural nets, etc. to recognize commercial magazine whole-issue structure. Our "semi-ground-truth" dataset -- as we'll only go down to typed page segments, and not down to baseline, word, and glyph boundaries, etc. -- will be a natural complement to the 880+ pages in the PRImA Magazine Layout Dataset seen here.

I am VERY excited to learn about the PRImA Research Center, its wonderful people, and the amazing tools they have created. I look forward to the progress that can be made through this evolving collaboration.

My 1st Post to the #cidocCRM SIG Mailing List

Jim Salmons — Wed, 13 May 2015 20:52:03 +0000

Today I felt bold or desperate enough -- actually some of both -- to prod a bit for a reply to a request for advice from a couple of my mentors in my #cidocCRM/#TEI Personal Learning Network. If you are successful developing a really good #PLNet (might as well mint a hashtag for on-going use), your loose group of mentors will, by definition, be EXTREMELY busy beyond your imagining, and the best won't suffer fools lightly. So evolving your #PLNet is always a balancing act. And a big part of that balancing has to do with your patience and the reasonableness of your expectations for what constitutes a good result or reply.

My pressing-inquiry to my #PLNet members revolved around what might be announced or open for discussion based on the preliminary agenda published ahead of the upcoming CIDOC CRM SIG meeting. My absolutely over-tasked #PLNet member found a moment to thoughtfully recommended that I join the CRM SIG mailing list (despite not being a SIG member), and make my interests and recommendations known prior to next week's meeting in Germany. He assured me that while he didn't make these SIG decisions, that my ideas and interest to get involved in the SIG would likely be well-received.

So with that encouragement, I joined the CIDOC CRM SIG mailing list and 'penned' one of my signature TL;DR unsolicited letters of introduction. I honestly don't intend them to be too long nor too deep. It is just that I feel once you step up on the soapbox and unload, the last thing you want to do is sit down and start running through all the things you wished you had said while you were up there. Right? With this context in mind, below the bar is my first post to the CIDOC CRM SIG Mailing List. (A subgoal, as always, is casting the net for connections to Kindred Spirits who are possible collaborators and/or mentors.)

If you get to the end of this note to the mailing list, I've provided a link to a moldy oldie from my Sohodojo days that provides additional insights about how to think about and use Personal Learning Networks to become whatever Empowered Individual you are intending to be. Oh, and there's an infographic to boot! :D

Hello CIDOC CRM SIG Members,

My name is Jim Salmons and my inter-related Citizen Science/History projects are @FactMiners (www.FactMiners.org) and @SoftalkApple (www.SoftalkApple.com).

We are working on two inter-related applied research initiatives; #cidocCRMgraph and #cidocCRMdev, as briefly introduced here: http://goo.gl/XZKkCE. For links to related #cidocCRM posts expressing our interests/insights, please see: http://goo.gl/dpbhPs.

BACKGROUND INFO

For our most complete “manifesto” of applied research interests, please see: https://goo.gl/3Vb0lO contributed and accepted into the CODE|WORDS series, and this GraphGist Edition of my #MCN2014 presentation: http://goo.gl/gS2FJk (especially the 2nd half of the embedded video which introduces, but does not dive too deeply into, our FactMiners #cidocCRM focus). And for some context and link to an unfolding #cidocCRM-related conversation at Schema.org in which I am trying to inject a metamodel-driven design POV, please see: http://goo.gl/x1DSAB. (Also, I am fortunate to have, and personally thank, the members of my #cidocCRM/#TEI Personal Learning Network who are helping me to fast-track my knowledge about the #cidocCRM and #TEI: https://goo.gl/skvhaj.)

CRM SIG MTG – NEW WEBSITE ITEM INTEREST & RECOMMENDATION FOR TEI P5 FORMAT FOR DEFINITION DOCUMENT

As to the upcoming CRM SIG meeting agenda item for next Friday morning to discuss a new cidoc-crm.org website, please note that @FactMiners has a long-standing public “hand raised” to either lead or contribute to this much-needed community project. If this is an open issue and the project’s nature and goals are to be determined through next Friday’s discussion, I would recommend that the new website be built HAND-IN-HAND with adopting TEI P5 as the semantic “format of record” for the CIDOC CRM Definition document. (MS-Word and PDF just won’t cut it moving forward when both humans and our software agents need equal access to this foundation document.)

While there are those who prefer/need an RDF encoding, there will be a growing group of folks like @FactMiners who will be best served by a human- and machine-readable edition of the Definition document “as is” without a complicated graph transformation applied (that can only further obfuscate existing graph-interpretation issues of the current state of the model). A TEI P5 encoding of the Definition document would not just be useful in its own right, but it could figure into an efficient workflow for the SIG in the future whereby the official public-facing website would be generated from (and updated by) releases of the official #TEI Definition document.

If the SIG would prefer not to select a particular lead or other sanctioned “official project” of the SIG, I would ask that you simply consider taking the existing cidoc-crm.org website and put it into a public GitHub repository so volunteer community members could fork it in response to a “CIDOC-CRM.org Website Make-over Challenge” with friendly competition encouraged by general public as well as a category for student/class projects.
There are a number of other active issues/ideas I would like to contribute to the current SIG conversations that will surely take place next week, but I don’t want to dilute my message about @FactMiners interest in contributing to the new website.

RECOMMENDATION FOR PREFERRED SOCIAL MEDIA HASHTAG - #cidocCRM

However, I will slip this recommendation into this self-introduction note to suggest that the SIG consider officially endorsing/recommending the use of #cidocCRM as the preferred hashtag for CIDOC CRM social media communication. My experience is that anything with any variation of tag that has a plain CRM or hyphenated #CIDOC-CRM etc., will attract the “900-pound gorilla” of Customer Relationship Management folks that subvert conversations or spike your followers with inappropriate bots or well-meaning lurkers. My choice of #cidocCRM is that the CIDOC aspect should not be the “shouty” part and that it serves as the “category discriminator” that will help keep our interests separate from “those other” CRM folks.

POSSIBLE SIG MEMBERSHIP?

Finally, given my intense interest in the #cidocCRM and not knowing how ICOM and/or CIDOC SIGs work, I am unsure whether I would be a prospective group member in that I am an unaffiliated, independent Citizen Scientist and on a limited fixed “retirement” budget.

Thank you for your interest and reading this notably long initial letter of introduction.

Happy-Healthy Vibes,
-: Jim :-

Jim Salmons
Twitter: @Jim_Salmons
www.FactMiners.org
www.SoftalkApple.com

If you would like to know more about Personal Learning Networks, I invite you to read a moldy oldie from my and Timlynn's Sohodojo adventure. Personal Learning Network or #PLNet was not what we called these networks fifteen years ago. In "The A-Team and The Sandbox" your #PLNet is your A-Team. The "sandbox" is what Timlynn and I have with our Citizen Science and Citizen History projects, www.FactMiners.org and www.SoftalkApple.com. In inforgraphic format, our post-cancer Portfolio Life looks like this:

#cidocCRMdev #FlyOnWall Comments Contributed at Schema.org

Jim Salmons — Tue, 12 May 2015 18:15:31 +0000

Context: This is the opening comment I made to a conversation at the GitHub repository for Schema.org where folks are considering a proposal or recommendation to "Add Exhibition as a subtype of Event".

My intent in contributing to this on-going conversation was to be a kind of #FlyOnWall reminder that the #cidocCRM -- the ISO standard Conceptual Reference Model for Museums and other cultural heritage organizations -- can be used as a process-oriented metamodel and not just as a descriptive ontology. This following opening statement can be found here. BTW, this comment can be read as a kind of "There's a pony in there somewhere..." piece to further reinforce the thesis of my "Witmore's Text..." Medium.com article.

Re @BarryNorton and @MiaLondon et al -- Whatever you do, Dan, PLEASE do it so as to be a #cidocCRM-compatible form.

Ontologists tend toward descriptive use of the #cidocCRM as a metaDATA standard when it is intended to also be a metaMODEL supporting its use to prescribe elementary building blocks (model elements, like parts in a LEGO blocks set) of software architectures. Until there is wider recognition of the SIGNIFICANT leverage that metamodel-driven software design can do for the Digital Humanities, the #cidocCRM will be as woefully under-utilized as it is currently under-appreciated. (Yes, as much great work is being done with the #cidocCRM, there is SO MUCH as-yet untapped potential in leveraging its metamodel nature.)

Why do I believe this? I was a lead in a Smalltalk-based skunkworks in IBM's Object Technology Practice doing "executable business model" frameworks in the 1990s behind closed doors of corporate consulting. Our work was inspired by David Gelernter's "Mirror Worlds" book, and was based on "self-descriptive" Smalltalk images that were compliant to an actor/role metamodel that objectified Process (an OOP heresy at the time).

Following my horrific cancer battle and a chance to stick my finger in the eye of the Reaper and come back for some Bonus Rounds, I find myself as a Wolf Child in the Wonderland of Digital Humanities' "Golden Moment."

When we were trying to do EBMs (executable business models) at IBM, we trolled the various IBM Global Services consulting practices for viable IRMs -- each practice was required to create an Industry Reference Model, AKA a metamodel. These IRMs ran the gamut from worthless tripe to "Wow!? Pretty good!"

I most enjoyed "pair programming" with my first ontologist, we called him "Doug the Librarian" because he "just modeled" and didn't code. Basically, what we were looking for in partners to do metamodel-driven software development were three things:

Subject Matter Experts (especially if they were verbal, thinkers, and open to being 'pushed' to clarity)
Explicit Models -- Black box expertise (wetware only) can't be executable without first being rendered in some explicit form of communication (usually words and images)
Source 'Instances' of these Explicit Models -- The best way to surface hidden assumptions and contradictions is to look at the delta of models that purport to be instances of the same metamodel

THE DIGITAL HUMANITIES DOMAIN HAS EVERYTHING NEEDED TO DO METAMODEL-DRIVEN SOFTWARE DESIGN AND DEVELOPMENT IN THE EXTREME! :D

And the #cidocCRM is the BEST opportunity I've seen to date to move in this direction.

Along these lines, I am working on #cidocCRM microservice workflows based on a 'self-descriptive' metamodel subgraph design pattern for LAM-based social games (front-end clients) and #cidocCRM-compliant collections management and scholarly editing back-end.

(BTW, a first step in this regard is to more rigorously express the #cidocCRM in pure-graph form to support vendor- and tech-neutral self-descriptive datastores.)

If anybody is interested in these ideas, please see: http://goo.gl/dpbhPs and http://goo.gl/gS2FJk, etc. at @FactMiners (www.FactMiners.org). This recent Medium.com article is the closest thing to a 'manifesto' on my applied research agenda: http://goo.gl/3Vb0lO.

In closing, I am an unaffiliated independent Citizen Scientist/Historian working from an Outsider POV. Inquiries to clarify ideas as well as explorations of opportunities to collaborate are most welcome @Jim_Salmons, @FactMiners, and @SoftalkApple

FactMiners' Fact Cloud and Witmore's Text as Massively Addressable Object

Jim Salmons — Thu, 09 Apr 2015 16:53:56 +0000

I've just published a long-read, Deep Weedsy piece on Medium.com entitled: "FactMiners' Fact Cloud and Witmore's Text as Massively Addressable Object". I welcome your reading and especially comments and questions.

My purpose in writing this article is two-fold:

As a backgrounder/profile -- a 'manifesto' of sorts -- that I hope Robert Miller of the Internet Archive will have a chance to read before we meet next week at #DPLAfest.

As part of my second post-cancer Bonus Round, my pipedream is to secure the interest and find external funding for a research fellowship to pursue our FactMiners and Softalk Apple Project applied research agenda from a position inside, and with collaborative support, of the Internet Archive to, in effect, join us in exploring the deeper levels of "massive addressability" of the Archive's digital collections, and
As a pubic letter of self-introduction to Michael Witmore, Jonathan Hope and the Computational Literary Linguistics community to explore "Rainman/Sherlock" collaborations as described in this newly published article and its related, earlier piece here.

I invite you to read the full article on your phone or tablet in the Medium app, or on-line in your browser.

Our #MCN2014 Presentation Exploring Metamodel Subgraphs - GraphGist Edition

Jim Salmons — Tue, 30 Dec 2014 18:14:48 +0000

We invite you to view the Neo4j-based "GraphGist Edition" of our recent #MCN2014 presentation about FactMiners and The Softalk Apple Project at the annual conference of the Museum Computer Network.

We again thank the Museum Computer Network Association and PICTION for sponsoring our Emerging Professional scholarship that enabled Jim Salmons and Timlynn Babitsky to attend the conference on behalf of our projects to learn, network with Kindred Spirits, and make our presentation to the museum technology community. The opportunity to attend this excellent conference was phenomenal and will be a source of enthusiasm and inspiration for our continued evolution of our grassroots Citizen Science/History projects.

What is the GraphGist Edition?

In the Open Source developers' world and most notably in the GitHub.com community, a gist is most usually written as a convenient document-based "how-to" or helpful short explanation about a useful snippet of programming code or a programming design or method that can be served in a "live example code" format. Folks interested in Python write Python-based gists, Ruby folks do Rudy-based gists, and so on.

In the Neo4j community, we have an incredible learning/sharing "live document" format called a GraphGist. The Neo Technology documentation team developed this clever server-based technology and documentation format standard to facilitate exploring and sharing ideas and techniques using Neo Technology's Open Source Neo4j graph database.

By writing to this standard and serving your GraphGist through the freely available gist.neo4j.org server, a writer can explore Neo4j's Cypher query language by showing and explaining queries that are then run server-side with the results being dynamically visualized in your browser view of the GraphGist. This document technology is obviously valuable for communicating to others in teaching and explanatory contexts. But GraphGists are also useful as an incredible self-learning and project-design documenting resource.

My #MCN2014 presentation includes a section that summarizes the core ideas of the metamodel subgraph design pattern. The "Where Facts Live" - Exploring the Metamodel Subgraph of a FactMiners Fact Cloud: GraphGist Edition provides a "live document" version of the four Cypher queries that I used to build the basic structure of a metamodel subgraph of Softalk magazine during my #MCN2014 presentation.

For those who may be interested in additional details and a "Hello, World" scale introduction to how a metamodel subgraph can be used for computational analytics as part of an Open Culture cognitive computing platform, see this two-part GraphGist on the metamodel subgraph pattern and this "Rainman/Sherlock" piece exploring the cognitive computing aspects of our project's mission.

Thoughts on CIDOC-CRM Classes: "Oops, who put all this Time Stuff in my Box of Things!?"

Jim Salmons — Tue, 04 Nov 2014 21:08:40 +0000

I am using the CIDOC-CRM – the Conceptual Reference Model developed by the International Council of Museums – as the primary domain reference model guiding design and development of the FactMiners social-game platform. In a recent post I looked at the Conceptual Reference Model from a "pure graph" perspective, re-imagining the CRM's Property Declarations as "just another" labeled subset of model elements, that is, as just another important subset of CRM Classes. In this post, I explore the "entity-ness" of the CIDOC-CRM Class Declarations.

Oops, We've Got a Lot of Time 'Stuff' in Our Box of Things!?

The approach I am taking on the design of the FactMiners platform is leveraging the expressive and flexible power of today's graph database technologies. This solution technology encourages me to look at the CIDOC-CRM as much as possible from a "full graph" expression. In addition to this graph-based interest, I believe my personal experience contributes a unique perspective in working with the CIDOC-CRM.

Twenty-odd years ago I was a thought leader and designer/developer in a "skunkworks" doing "executable business model" software frameworks in Smalltalk as part of the Object Technology Practice of IBM Global Services. Through this work we did groundbreaking work "objectifying" Process, using an agent-based perspective, and driven by strict adherence to a rather simple but powerful role-actor metamodel. A "smart desktop" framework dynamically generated "application" views on the executing business model. Change the business model and the "applications" dynamically changed. We had an opportunity to "troll" the IBM Global Services consulting practices looking at their IRMs – their Industry Reference Models. We were looking for other practice collaborators who had a decent IRM and customers who might be interested in, or better yet, need exploratory business modeling services.

So when I looked into the "pile of stuff" that is the CIDOC-CRM, I had a whole lot of context and tactics for sorting stuff out and "seeing" what's there. Recently, as I poured over the CRM Definition with an eye toward importing the Property Declarations into a Neo4j graph database, I had some "A-ha!" Moments where I think I see what might be the biggest source of frustration folks have when trying to "dip into" (that is, to explore and understand) the CIDOC-CRM.

One of the distinguishing characteristics of the CIDOC-CRM is its "object-oriented" foundation. The OOP influence in the expression of the CIDOC-CRM is significant and useful. However, I believe that a less-technical reader will not see – or worse, justifiably be confused by – the important OOP-modeling distinction between "Thing objects" and "Activity objects" – to use the most general of Class names found in the E2 Temporal Entities and E77 Persistent Item branches of the class hierarchy.

Objectifying process as a first class object is a rather subtle OOP technique with significant implications. The current CIDOC-CRM Definition text document, the on-line class hierarchy diagram, and the various subsystem diagrams seriously underplay this important design distinction. I believe that if the official CRM Definition text and associated graphical diagrams more fully "set the stage" for understanding the importance and utility of this underlying "Big Picture" nature of the CRM, the motivated exploratory user might have a more successful experience.

Let's Take A Quick Look...

As you can see in the following screenshot collage, the official 'Definition of the CIDOC Conceptual Reference Model' presents the CRM Class Declarations as a prefix-sorted master list. The Modelling Principles section jumps immediately into some very deep and subtle distinctions of monotonicity, disjointness, extension, etc. while overlooking this considerable distinction between the E2 Temporal Entities and E77 Persistent Item branches of the hierarchy. Only two summary lists visually show the deeply indented hierarchies of both the Class and Property Declarations. And the explanation of this significance is barely discussed.

What jumps out at you is that the current CRM Definition document does a great job of being the 'Volume 2. Reference' of a two-volume set where the first volume provides the Big Picture perspective and example-based Getting Started material. This "missing volume" is a necessary complement to the strict "just the facts" Definition reference. Unofficial community contributions help to fill this void – notably, e.g., Dominic Oldman and the CRM Labs' "CIDOC-CRM: Primer V. 1.1" (PDF). However, the official www.CIDOC-CRM.org provided reference documents should include such a complementary introductory volume as part of the Definition set.

This "missing" introductory volume of the CRM reference documents would surely incorporate graphical diagrams similar to those currently available on the official CIDOC-CRM.org website. But a quick look at the available diagrams – as hard as I know it was to create and produce them them – confirms that the current resources dramatically underplay this important distinction between Thing Objects and Process (Temporal) Objects.

The current text reference and class hierarchy diagram do little to highlight the important distinction between the significant partition of the CRM model elements into 'Thing Objects' and 'Process Objects'.

The situation doesn't become any clearer when we look at the numerous subcomponent model diagrams on the CIDOC-CRM.org website. Again, I know how hard it was for someone to draw and validate these images. These diagrams are super helpful as far as they go. But their format seems more constrained by the diagram drawing tool used rather than there being an explicit model diagramming standard driving the current diagram presentation format.

Here, for example, is the Object Association Information component diagram. I picked this as it is relatively sparse and includes a good mix of 'Thing Objects and 'Process Objects.' This part of the model explains how Things and People can be found at certain Times in specific Places participating in discrete Events, etc.

The physical arrangement and IS_A superclass relationship double-line arrows reflect this Thing/Activity distinction among the CRM model elements, but this is a very subtle visual distinction. And as with all the current component diagrams on the official website, this diagram is a static image. There are no interactive links between the graphical Class nodes and Property edges and their respective entries in the official Definition text.

My own frustration at diving into and around the www.CIDOC-CRM.org on-line reference material has encouraged my desire to contribute an updated and more interactive reference resource for the model to the CIDOC-CRM community. Such a more exploratory resource could be a step-wise generalized contribution of our project beyond the specific focus of our work on the FactMiners platform. Here, for example, is what I did to explore some potential along these lines...

I started with a fresh copy of a Neo4j graph database with the Class Declarations as nodes and Property Declarations as relationships (between nodes). (The GitHub for this 'seed' of CIDOC-CRM model elements in a Neo4j database is here.)
I did a Cypher query to add a 'Persistent' and 'Temporal' label to each class based on its membership in the E77 or E2 branch of the class hierarchy.
I then did a query in the stock Neo4j browser to return and visualize the model elements (CRM Entities) associated with the Object Association Information component.
I next dragged the nodes of the "bouncy ball" browser graph visualization to more closely resemble the static diagram of the target diagram.
Finally, I clicked on the E7 Activity node to pop up its node properties (not to be confused with CRM Properties) to show how the full CRM Definition entries are in this Neo4j database... and then I did this screenshot.

While this stock browser visualization is too "one-off" to be truly useful, this does show the impact of something as simple as subset coloration (or other containment revealing visual queue) to help reveal the underlying design of the CRM model elements. And although the static nature of the screenshot does not reveal it, the ability to examine full Definition entries within the exploratory diagram is another step in a good direction to consider for the official CIDOC-CRM documentation and its companion website. If this much functionality can be gained from one-off use of available generic tools, imagine what we could accomplish by building a CIDOC-CRM reference resource with a full RESTful and public web interface built on a platform based on Neo4j + Structr + KeyLines (or Alchemy.js, Linkurious, or similar client-side visualization layer).

Exploratory Decomposition of CIDOC-CRM Graphical Diagrams

Continuing our thought experiment imagining an expanded set of CIDOC-CRM reference documents, I would encourage an effort to encapsulate the important structural design distinctions through a logical/functional "exploratory decomposition" graphical style. I simply don't have all the answers as to what this means fully, but I can provide an example based on my own thought experiment along these lines.

I mentioned that my prior "executable business model" experience was helpful to my understanding of the CRM. I also mentioned how the Smalltalk framework we built was agent-based, objectified Process, and was based on a "ruthlessly simple" top-level metamodel. I took a UML (Universal Modeling Language) Class model that is very similar to what we did in that Smalltalk skunkworks, and I overlaid the high-order CRM Entities that have an obvious model element alignment with this process-oriented UML Class model.

At one level, this diagram looks too simple to say all that much about a non-trivial software architecture. But appearances are deceiving when it comes to such software design diagrams. This UML diagram says a LOT about an agent-based role/actor software architecture. And having written software frameworks implementing such architectures, I know that the beauty and the "devil" is in the details suggested by such high-level diagrams. I know, for example, how extensive the OOP-programming decomposition of the Activity and Task objects can be to implement the required interfaces and functions of such a high-level diagram. Yet as complex and diffuse as the implementation decomposition of these high-level model elements may be, well-linked logical decomposition of such inter-linked diagrams of the full model can keep the reader in context to avoid confusion or misunderstanding.

This "Just Enough, Just In Time" exploratory presentation style is how we need prospective CIDOC-CRM users to be exposed to the full CRM model. We need a rich set of interactive diagrams that keep the reference-reader "in the moment" and in context. A "double-click dive" into the E7 Activity node, for example, would reveal the decomposition of Activities into the various modeled elements including E11 Modification, E10 Transfer of Custody, E86 Joining, E87 Leaving, E79 Part Addition, etc. In each case, a context-retaining diagram specific to the "leaf" class being investigated will not only be helpful for general model learners, but will be invaluable for software architects intending to use the CRM as a metamodel constraining a full system instance design.

I believe that there are far more developers today who are "comfortable users" of object-oriented programming technologies than there are those who are comfortable with designing and building systems based on advanced OOP architectures, especially ones that objectify process. For this reason, I believe that many mainstream programmers would find the current model reference resources insufficient to design and build full CIDOC-CRM based systems. Not that it can't be done... just that there would necessarily be a lot of "private mental concept transformations" required to get the job done.

Given this belief that there are more casual users of OOP technology than there are advanced OOP designer/developers, the proposed "missing" first volume of the CIDOC-CRM Reference documents should be envisioned as a Developer's Guide to CRM-compatible system design and programming. The non-technical/non-developer would find the overview and Big Picture content of such a guide useful. An exploratory learner can simply stop his or her reading of this introductory guide once the information goes deeper than needed for the learner to collaborate as a subject matter expert on a team developing such systems, or to assess the model when making a recommendation about possible adoption of the CRM by the prospective user's institution. Such a proposed Developer's Guide, however, would provide an invaluable "easy on ramp" for software architects and developers tasked with building CRM-compatible systems.

At this point, I can't go further without some non-trivial time and effort to provide additional specific examples of where we could go with CIDOC-CRM reference documentation. I do believe, however, that this post already sufficiently suggests why we might want to do this and how this effort might be approached.

As always, I appreciate comments and questions,
-: Jim :-

Thoughts on a Graph Representation of the CIDOC-CRM: Property Declarations

Jim Salmons — Sun, 02 Nov 2014 00:12:09 +0000

I am having an interesting time looking at the CIDOC-CRM – the Conceptual Reference Model for museums developed by the International Council of Museums (ICOM). In particular, as generally described in my last post, I am looking at the CIDOC-CRM with a "pure graph" lens on a metamodel developed with an object-oriented perspective.

The interesting part of my exploration has to do with bringing the "ruthless simplicity" of a graph expression to the interpretation of the CIDOC-CRM for the purpose of representing it as a Referenced Model within the metamodel partition of the graph database as part of the FactMiners' Digital Humanities-based social-game platform. I've written two complementary Neo4j-based GraphGists that explore the design pattern of an "embedded metamodel subgraph" as a means to build a fine-grained "Fact Cloud" of the referential content of the 48-issue collection of Softalk magazine covering the early history of the microcomputing revolution.

In this post, I'll take a first look at how CIDOC-CRM Property Declarations can be beneficially cast as "First Class Citizens" (AKA Nodes) in a graph model of the CIDOC-CRM. I am still in the very early stages of CIDOC-CRM exploration, so take these observations as public research notes.

The Challenges and Potential of the CIDOC-CRM

An interesting dimension of the CIDOC-CRM is that it intentionally tries to be a comprehensive conceptual model (AKA a metamodel) for not just museum artifact description, but to cover the active process of artifact curation and preservation as well as broader realms of collection management, etc. In this sense, the CIDOC-CRM goes where most cultural artifact ontologies rarely go, that is, into the realm of incorporating process along with structure-oriented model elements from which to construct our CRM-compatible model instances.

That's what a Conceptual Reference Model or metamodel is used for, at its most basic level. It's a set of model elements (parts to build a model out of) and a set of instructions that constrain how you can put these parts together. Through this model, we can share an understanding about what some domain of activity and expertise is about. That understanding is to be used, in our case, to design software systems to implement this shared conceptual (mental) understanding of what needs doing and how to do it. (I know this is not the only reason for having a comprehensive conceptual reference or metamodel, but it is the compelling reason for this exploration and discussion.)

As is often the case, industry reference models are rarely used to drive down to the granularity of "executable" in the sense of software being designed strictly to the, let's call it "semantic expressibility" of the metamodel. In other words, these conceptual reference models are most often used loosely to guide human-to-human ("mind-meld") conversation and modeling-based decision-making. But there is a lot of room for what are in effect "uncontrolled individual model transformations" (AKA we developers think about how to implement the model and then write some code that approximates our personal understanding of this shared vision). So if during implementation of metamodel-driven design there are "gotchas" or logical inconsistencies or just plain "temporal anomalies" (a big model done by committee over a long period of time), these issues are not too often show-stopping problems as we "one off" resolve these issues by "coding band-aids" as we go along.

But one of the goals that the CIDOC-CRM SIG group members aspire to encourage and support is research-based inference and deductibility – text- and image-based computational analytics – for CIDOC-CRM compatible datastores and associated #LODLAM web services. In other words, "close enough for horseshoes or hand grenades" may be good enough for many cases of using a conceptual reference model for indirect reference, but this is not good enough if we are going for fact- or inference-discovery and validation via metamodel-constrained computational analytics.

So let's look at the most obvious case in point; Property Declarations as "faux" relationships between CRM Classes.

CIDOC-CRM Property Declarations as "Shortcut" Classes

A typical CIDOC-CRM graphical diagram of key parts of the model shows the graph-based shared understanding of how we're "thinking about" this museum conceptual model. The "object"-like things are drawn as nodes and have "E-prefix" names that reflect their "IS_A" (object-oriented inheritance) descent from the "E1 CRM Entity" root of the CIDOC-CRM Class Hierarchy. The relationships (or edges in a pure graph sense) between Classes (AKA our graph-based Nodes) are defined as CIDOC-CRM Properties. CIDOC-CRM Properties have a "Pnum-prefix" name and have the additional facet of "bidirectional/reflexive naming" -- e.g. P67 "refers to/is referred to by" providing the appropriate human-readable interpretation of the named relationship based on which "end" of the relationship you want to anchor your mental understanding of the described relationship.

To understand how this current view on the CIDOC-CRM is problematic to a "pure graph" expression, let's look at a particularly interesting cluster of CIDOC-CRM model elements (both Classes and Properties) that describe the "Image Information, Objects and Carriers". I've highlighted the aspects of this diagram of the CIDOC-CRM definition that cannot be expressed in a basic graph representation.

This subset of CIDOC-CRM model elements is especially interesting as it gets to the core of our FactMiners' design interest – and that is, "piercing the veil" to move our Fact Cloud coverage from artifact description to deep analysis by modeling the artifact's representational meaning. In our case, we require a metamodel sufficient to cover the complex structure of a commerical magazine while providing the granularity to model – based on CIDOC-CRM-compatible form – the 48 monthly snapshots of the cacophonous "Open World" of activity at the dawn of the Microcomputer Revolution as depicted in the incredible content of this historic magazine archive.

The eagle-eyed "graphistas" among you will notice a show-stopping gotcha. Those double-line arrows from relationship-to-relationship violate the essential "ruthless simplicity" of a pure graph model. A graph is just nodes and relationships, with relationships being the edge/line links BETWEEN nodes. There are no relationships between relationships in basic graph theory.

Fortunately, this is not a show-stopping problem. It just suggests that the view we're given is more a "mental shortcut" for a graph expression that models the relationship as a distinct type of Class. In this way, a "node-ified" CIDOC-CRM Property can be modeled with respect to its "IS_A" subproperty/superproperty relationships as well as support the "Pnum.1" idiom where property specializations can be expressed through "mode of X" relationships to "E55 Type" entities, etc.

In other words, here is how we can look at a portion of the "Image Information, Objects and Carriers" model with "node-ified" CIDOC-CRM Property Declarations:

In this diagram, the "subset-membership containment areas" are a visualization of, in Neo4j graph database-terms, a label-based conceptual type. That is, our metamodel of the CIDOC-CRM contains Nodes that are of type "CRM Class" and a collection of Nodes of type "CRM Property." These two labeled subsets are emphasized in the graphic by node color in addition to their containment in subset-membership labeled boxes.

Now that we have a "node-ified" Property "P138 represent/has representation" represented in our graph as a node, we can create its "subproperty of" relationship to "P76 refers to/is referred to by." This "subproperty of" relationship, by the way, can be thought of as a specialization of the "IS_A" relationship underlying the object-oriented CIDOC-CRM Class subclass and superclass inheritance relationships. Note, too, that we can easily create the "Pnum.1" relationships that provide specialization of a CRM Property.

[Note: The above diagram is decidedly high-level. A full rendering of a "pure graph" expression of the CIDOC-CRM would show labeled domain/range relationships, cardinality, etc. These fine-grained aspects of the model are left off here to keep the explanatory diagram cleaner.]

Okay, but so what? Why "node-ify" CRM Properties?

At some level we could think, "Okay, so what?" Aren't we just nit-picking over the details of the metamodel representation for communication and system design discussions. For many, yes, that's true. But the FactMiners LAM-based social-game platform is being designed with an underlying cognitive computing perspective. We need our metamodel expression of the CIDOC-CRM to be machine-executable.

This aspiration for deep computational analysis of cultural artifacts is also one held by members of the CIDOC-CRM Special Interest Group. Dominic Oldman – a CIDOC-CRM SIG member, Principle Investigator of www.ResearchSpace.org, and IT exec of The British Museum – writes in the latest version of "The CIDOC Conceptual Reference Model (CIDOC CRM): PRIMER" (my emphasis added):

"The most important kinds of computer-based reasoning the CRM can support are generalisations of relationships and deductions from highly indirect relations such as what parts have in common with their wholes, what wholes inherit from their parts and what is transferred across meetings and processes of derivation. These are not meant to replace scholarly conclusions but to comprehensively detect facts relevant to answer research questions. Besides others this ensures that highly specialized knowledge stays accessible to generic questions regardless the specificity of representation."

My belief is that a fully-realized graph expression of the CIDOC-CRM will not just support the kind of "fact-checking" and discovery that Dominic asserts in the quote above, but will move CIDOC-CRM-based computational analytics into the realm of significant first-class, fine-grained, traceable, interpretive scholarship that is only just beginning to be imagined based on the rapid advances in Deep Learning, Natural Language Processing, Image Scene Recognition, and other aspects of the explosion of advances in what is being loosely corralled by the term "Cognitive Computing."

While my example is ruthlessly simple for the purpose of introductory explanation and is not CIDOC-CRM-specific, the first part of my two-part GraphGist demonstrates how generalized query-based fact discovery and validation can be performed on a "self-descriptive" Neo4j-based graph database. And closely related to this GraphGist is the research agenda described in my "Rainman Meet Sherlock" post.

I welcome your thoughts and/or questions.

On "Node-ifying" Relations in a Neo4j Graph Database of the CIDOC-CRM

Jim Salmons — Sat, 25 Oct 2014 19:13:04 +0000

I am very fortunate to be developing a Personal Learning Network with some amazingly brilliant people helping me to fast-track my learning and use of the ICOM CIDOC-CRM – that is, the ISO standard Conceptual Reference Model (i.e. a metamodel) for museum cultural artifact description, preservation and associated collection management developed by the International Council of Museums (ICOM).

As the learner of a PLN network, you look for opportunities to "give" as well as "take" as we are too often on the taking side of things. As in our taking of valuable time and energy from our PLN mentors. So looking for such an opportunity to give something back, I mentioned to Dominic Oldman -- of the British Museum, its www.ResearchSpace.org project, and member of the CIDOC-CRM SIG -- that I would like to make a community contribution of a more intuitive and easier-to-navigate exploratory viewer for the CIDOC-CRM Definition reference. He said that sounded like a good idea. So this post is related to that quest.

Some quick background... I am designing the FactMiners LAM-based (Libraries, Archives, and Museums) social-game platform based on an "embedded metamodel subgraph" design pattern. To make the output of our FactMiners gameplay useful, not just for casual visitor engagement, but for scholarly and scientific research, we have adopted the CIDOC-CRM as our primary Reference Model in the META partition (subgraph) of the FactMiners Fact Cloud. So while we are building a social-game platform on the one hand, it will also be a credible CIDOC-CRM compatible platform that can be easily and modularly extended into a full-featured digital collection management system on the other.

It is our hope that the FactMiners social game platform will find broad use in grassroots digital Citizen History projects due to its "seriously fun" gameplay and associated player community. But with our underlying CIDOC-CRM compatible "gameflow", we will produce digital cultural heritage collections that can be both immediately published to evolving #LODLAM (Linked Open Data in Libraries, Archives, and Museums) standards, as well as easily transferred to more sustainable long-term preservation within fully-curated institutional collections.

"Node-ifying" a Graph Relationship -- DIY 2-D Hypergraph!?

One aspect of my metamodel subgraph design pattern has to do with a graph transformation that can be viewed as "node-ifying" a relationship. That is, in order to more completely model a relationship in a target graph, we explicitly model the target relationship as a node in our metamodel. I think of this transformation as an elementary "2-dimensional hypergraph" in that the "node-ified" relationship and its domain and range relationships are a subgraph decomposition of the target graph's relationship. The cartoon graphic accompanying this post more easily demonstrates this mapping/transformation.

If you look at the official 'Definition of the CIDOC Conceptual Reference Model' (PDF), you see the two primary content divisions of the Class and Property Declarations. The basic building block model elements are described as class/nodes and property/relationships. When graphical diagrams of the CIDOC-CRM are provided, this "class as node" and "property as relationship" concept is clear.

But a closer reading of the Property Declarations shows that the "devil is in the detail." That is, in order to do a graph model of the Property Declarations, we need relations to have relations. The most obvious examples of this being the superproperty and subproperty relationships, and the "Pxx.1 has type" properties. At first this realization hit me with a, "Hmmm, this could be trickier than I thought..." Moment.

Things get complicated very quickly when you cross over into "hyper zone" with graph thinking. And more importantly, the incredibly powerful and widely available Open Source graph databases like our choice, Neo4j, do not support relationships having relationships. But then I clicked on the idea that using the "node-ifying" relationship pattern, like what I'm doing in the FactMiners META subgraph, could work well in building an exploratory viewer for the CIDOC-CRM Definition.

To bring the CIDOC-CRM Definition "to life" in an exploratory viewer, we'll need to provide "shortcut" (i.e. collapsed) visualizations of the model that make the resulting interactive graphical diagrams less cluttered and more intuitively meaningful.

The most awesome platform I can imagine on which to build this exploratory viewer is a Neo4j-based Struct-powered web and mobile app with client-enhanced views based on the equally brilliant KeyLines graph visualization product. As a next step in my quest to provide the CIDOC-CRM community with a new model viewing resource, I will be asking Cambridge Intelligence's U.S. developer rep, Corey Lanum, to check out this post. Can the powerful visualization features of KeyLines be harnessed to provide a "collapsed/expanded" visualization for my CIDOC-CRM Explorer that defaults to the "relationship as relationship" view with a double-click event toggling in and out of the expanded "node-ified" full-path view?

I am anticipating that KeyLines can do what I need. This seems like just the kind of view transformation mapping that could/should be pushed to a dynamic visualization level decision rather than be made at the database design level. I will update this post as this exploration unfolds...

For more, see my initial explorations in this GraphGist: The CIDOC-CRM in a Neo4j Graph Database.

Citizen Science as Means to Increasing Diversity in the Library and Information Science Profession

Jim Salmons — Tue, 14 Oct 2014 07:18:58 +0000

This is an open letter of introduction to an amazing group of Knowledge Allies in the American Library Association (ALA) Knowledge Alliance Program. In particular, I am addressing the sub-group of Allies who have a stated interest in Digital Preservation; Melissa Cardenas-Dow, Patricia Hswe, Somaly Kim Wu, Sally Ma, Michael Elliott McWilliams, and Kai Alexis Smith.

I learned about the Knowledge Alliance Program in the current issue of 'Archival Outlook', the newsletter of the Society of American Archivists. The 'Diversifying the Library and Information Science Profession' article is a real 'call to arms' to all of us to do what we can to introduce talented minority STEM students to the opportunities and support available upon focusing on Library and Information Science (LIS) studies at undergraduate and graduate levels. (Click here for the beautiful full issue PDF and see page 10.)

As the article cites, in a study by the Association for Library and Information Science Education not much has changed since 2001 when students of color comprised 12 percent (504) of the 4,953 LIS graduates. From 2000 to 2010, the number of employed underrepresented racial and ethnic credentialed librarians has increased by only 1 percent, according to the ALA's Diversity Counts report.

The ALA's recognition of the importance of increasing diversity within the Library and Information Science (LIS) profession is more than commendable, it is essential. Because many of us know, "Librarian" is just the "Clark Kent" name for the Information Science Superheros that graduates of LIS programs will be in the emerging Linked Open and Big Data World of the 21st Century.

As talented minority STEM students recognize that their studies can lean toward the "Information Science" side of the LIS field of study, a whole range of creative passion-driven Life Opportunities emerge where, "Clark Kent, Librarian" might have seemed too-limiting a career aspiration.

From Hackathon Through Citizen Science/History to LIS Education

So having read this eye-opening article, I got to thinking about how FactMiners.org might collaborate with our #Play2Learn buddies to help bend that LIS enrollment curve upward.

At the moment, my wife and I have two projects in the semi-finals of the Ashoka/LEGO #Play2Learn Re-imagine Learning Challenge. FactMiners and The Softalk Apple Project are grassroots, independent Citizen Science/History projects. As entrants in this Challenge, we are involved in growing collaborations with fellow social entrepreneurs doing all kinds of amazing things in the Kindred Spirit of re-imagining what education needs to be in order to prepare, not just kids, but everyone for what it means to be an Empowered Individual in the 21st Century.

One social entrepreneur doing amazing work through his Qeyno Labs "Hackathon Incubator" is Kalimah Priforce. Kalimah has nurtured Qeyno Labs into an "opportunity-generating ecosystem" that brings together its own amazing bunch of people and all manner of partner organizations collaborating -- using their own description -- "to harness the interests of high potential youth from low-opportunity settings into STEM career pathways."

A hackathon is an in-person, mentored, intensive, learn-by-doing experience to teach interested kids how to code by building an app in a matter of a couple of days or a weekend. As I imagined the challenges that Kalimah's team must face, it became pretty clear that being the inspired start of a "STEM career pathway" implied further steps in follow-through from that inspiring start. These pathways would be composed of a series of "hand-offs" of mentored relationships that help an inspired Hackathon graduate reach his or her potential.

In some cases, the Hackathon-built app has its own "follow-through" power and creates the opportunity to build a business or developers' community around continued improvement of the app itself. But in many cases, the Hackathon will light a flame that needs to be fanned and its heat directed to the individual interests of the Hackathon graduate.

That's where Citizen Science/History projects and the Knowledge Alliance Allies come into play!

FactMiners and our associated Softalk Apple Project are already exploring a "mentored pair programming" collaboration as part of our mutual involvement in www.DIYPlanetWalk.com, a self-help experiment in "bundled crowdfunding" by a group of Kindred Spirits in the #Play2Learn Challenge. If we are successful in our Kickstarter campaign, we'll have a good pilot for creating a "first step hand-off" in what can then lead to that "STEM->LIS" interest-focusing pathway for inspired Hackathon grads. This would seem like a great "win-win-win" all around.

What we would like to propose to our Kindred Spirit Allies in the Knowledge Alliance program is that you work with us to create a good communication channel and "smooth hand-off" so that grassroots Citizen Science/History projects can become a valuable "feeder system" for directing high potential youth from low-opportunity settings into Library and Information Science careers.

Perhaps something as simple as a new page on the Knowledge Alliance website with a list of "registered/interested" Citizen Science/History projects that are available for both paid and volunteer internships or Open Source contract development projects. Through our initial collaboration, we could develop the processes and materials needed to allow easy expansion of this network of creative career-shaping collaboration.

Next Steps

@ALALibrary @librarycourtney - Turns out @KnowledgeAll is '.net' and not '.org' - This meant for your Allies :-) http://t.co/tCT2mBPxIx

— FactMiners.org (@FactMiners) October 24, 2014

I will tweet to bring this blog post to the attention of the Allies at the Knowledge Alliance. I will also create a new user account as someone "interested in exploring the library profession" on the KnowledgeAlliance.org website. Through whatever means of communication works first, I look forward to learning more about the Knowledge Alliance and how we might work together to bring some talented young people into the Library and Information Science profession. We would love nothing more than to help young minds see the Information Science Superheros so many of us mistake for "typical librarians."

FactMiners Jim Awarded Emerging Professional Scholarship to Attend #MCN2014

Jim Salmons — Thu, 02 Oct 2014 23:17:42 +0000

I am beyond happy to announce that I have been awarded an Emerging Professional Scholarship to significantly support our attending and presenting at the prestigious Museum Computer Network 2014 (#MCN2014) annual conference next month in Dallas, Texas. While it is an honor to be recognized (at my age and as an unaffiliated Citizen Scientist/Historian) with this award, it is truly the financial impact of the award that mean so much to me and FactMiners "co-host" (my soulmate/wife) Timlynn Babitsky.

Timlynn and I will be attending and presenting on behalf of FactMiners.org and The Softalk Apple Project, our First Of A Kind "sandbox" for development of the FactMiners' cognitive computing-based social game platform for Libraries, Archives, and Museums (LAMs). You will find our proposals for our 30-minute presentation and demo here. Our demo will be on Demo Day, Friday, November 21st, and our presentation will be one of three in the panel "Shoestrings, Subgraphs, and Sharing: New Design Paradigms for Museums" scheduled for Saturday, November 22nd.

Timlynn has already signed up to be a Conference Volunteer. The Emerging Professional Scholarship is quite generous and significantly appreciated:

one full conference registration
complimentary attendance at ANY of the the pre-conference workshops (these are GREAT and not cheap)
a $400 travel/food stipend!
complimentary hotel room in the conference hotel
complimentary 1-year Emerging Professional membership in the Museum Computer Network! :-)
plus... a Cocktail Meet & Greet for Scholarship Recipients (great networking)

So, yes, we are super-stoked. As an independent, unaffiliated Citizen Science project, this kind of support means everything to us. THANK YOU MCN FOLKS! :D :D We will see y'all in Dallas!!!