FactMiners.org - News

Softalk Apple Is First Collection at Internet Archive with FactMiners' MAGAZINE Ground Truth Storage Metadata

Jim Salmons — Mon, 06 Mar 2017 19:48:23 +0000

FactMiners and The Softalk Apple Project are excited to announce that The Softalk Apple Project's digital collection of the Apple edition of Softalk magazine is now included in both "The Computer Magazine Archives" and "Magazine Rack" collections at the Internet Archive. Our projects also were granted full admin rights to the Softalk Apple collection at the Archive in support of our applied research.

And the first BIG NEWS made possible by our having full admin access is that The Softalk Apple Project collection is the FIRST (and so-far ONLY) digital magazine, newspaper, or serial publication at the Archive to provide XML-based FactMiners' MAGAZINE #GTS (Ground Truth Storage) metadata files for each issue of the magazine as well as a "master file" for the entire publication/collection!

"Ground Truth Storage" is a term that image-analysis and text-mining researchers use for metadata files that are human-curated and validated as (close to) 100% accurate as possible. The #GTS format that we developed at FactMiners is based on an 'ontological stack' of the #cidocCRM (the International Council of Museums' Conceptual Reference Model for Cultural Heritage), FRBRoo (the IFLA's Functional Requirements for Bibliographic Records) and PRESSoo (the IFLA model for serial publications). Rather than focus on the within-page ground truth of individual page layout and text recognition, the FactMiners' MAGAZINE metadata format incorporates a comprehensive, publication-wide metadata model that integrates the complex Document Strucure and Content Depiction models.

In an effort to keep our project collaborators and supporters informed, we made two short demo screencast video updates about our progress developing the Python-based ppg2leaf_ferret metadata discovery and validation tool:

with the second update showcasing our generalization of the ferret to handle bottom-margin page number spotting. The issue we quickly explore is the famous August 1981 issue of Byte magazine all about Smalltalk:

To take a look at the initial iteration of the FactMiners MAGAZINE #GTS (Ground Truth Storage) format metadata files at the Internet Archive, see here:

https://archive.org/download/softalkapple/softalkapple_publication.xml

for the publication level MAGAZINE #GTS file, and here:

https://archive.org/download/softalkv1n01sep1980/softalkv1n01sep1980_mag...

for an example of the issue level MAGAZINE #GTS metadata file.

Keep in mind that this initial publication of our MAGAZINE metadata files is very "thin" at the moment, with mostly empty placeholder tags that will be filled in with full models and associated datasets. The XML Schemas for the MAGAZINE format are published and available to all researchers via the FactMiners website. See the XML header of the above metadata files for the standard XML schema location reference to these files.

The individual issue level MAGAZINE #GTS metadata files include a "ppg2leaf_map" that guarantees the relationship between Softalk's print page numbers and their respective "leaf" images in the digital copies at the Archive. While our MAGAZINE files are admittedly lean at the moment, The Softalk Apple Project already has extensive data "in the can" -- being curated Advertisers Index, mastheads, Table of Contents, and lists of Companies, People, Products, etc. who made or were covered editorially in the magazine. We are currently writing the Python scripts to generate the XML metadata that will begin populating the publication level model and dataset metadata. Issue-specific subsets of our data will also be included in the issue level metadata files.

The publication of the these MAGAZINE #GTS files is the subject of FactMiners' first paper submitted to #DATeCH2017, and the ppg2leaf mapping found in the issue level files is the subject of our second paper to this EuropeanaTech Digital Humanities research conference which is scheduled to take place in Germany in early June.

The FactMiners #GTS format is being evolved as a resource to support eResearch and machine learning at the Internet Archive. As always, comments and questions are welcome. Even better, we welcome volunteers who would like to become involved in our #CitizenScience and #CitizenHistory projects, FactMiners and The Softalk Apple Project. To express your interest feel free to contact us through this website or via social media channels.

For those interested in reading pre-review PDFs of our #DATeCH2017 submissions, they are available to those with ResearchGate.net access at "Ground Truth & Softalk Magazine: Using Aletheia Web Edition to do FactMiners’ Text-mining" and "Print-Page Number to "Leaf" ID Mapping in Support of eResearch and Machine-Learning at the Internet Archive". Others interested in our applied research may request personal communication copies via the contact form on this website or through any of the social media channels in which we are active.

FactMiners & PRImA Partner on Knight News Challenge Entry

Jim Salmons — Sat, 17 Oct 2015 21:11:43 +0000

FactMiners is immensely pleased to announce our collaborative entry in the current "Data Edition" of the Knight News Challenge. With encouragement of our the Internet Archive, we have proposed an innovative idea and project implementation to partner with PRImA, the Pattern Recognition and Image Analysis Research Center to "Turn Text Soup into Smart Data in Newspaper & Magazine Archives".

The Knight News Challenge is unique in that it encourages entries from individuals as well as organizations, and international partnerships are no problem. So this potential source of funding for our applied research at the Internet Archive is most appreciated. FactMiners (so far) consists of two post-cancer #PayItForward Citizen Scientists in Cedar Rapids, Iowa, USA, and our two fellow Kindred Spirits on this project -- Apostolos Antonacopoulos and Christian Clausner -- are world-class research scientists at PRImA at the University of Salford, Manchester, UK.

We hope you have a few minutes to visit our entry to learn more about our project. We've prepared four "silent Ignite Talk" video slideshows to showcase our project. Your comments, questions, and especially, your "applause" (AKA a "like") are most welcome and appreciated.

FactMiners & Softalk Apple Project to Host Global Collaboration Day Event!

Jim Salmons — Thu, 10 Sep 2015 19:42:19 +0000

Psssst.... this is just a quick heads-up ahead of the official announcement forthcoming...

FactMiners & The Softalk Apple Project are pleased to announce that we will host a Global Collaboration Day Event "all day" on September 17th.! :-)

Details of our event will be forthcoming and announced here as an update to this announcement. Time is short, so we're in crunch mode to be ready for this fun event.

On September 17th, students, teachers, and organizations – including grassroots Citizen Science and Citizen History projects like ours – will celebrate global collaboration! On this day, experienced global educators and professionals will host connective projects and events and invite public participation. The primary goals of this whole day event are to demonstrate the power of global connectivity in classrooms, schools, institutions of informal learning and universities around the world, and to introduce others to the tools, resources and projects that are available to educators today.

At FactMiners and The Softalk Apple Project, we believe in the power of connected learners... Let's show others what globally connected learning looks like on September 17th! Keep an ear out for our and other Global Collaboration Day events. And if you are a connected learner organization of any stripe, we encourage you to join in and host a virtual event. Everyone everywhere learning together... start with a day, make it a habit.

FactMiners Partners with Internet Archive to Accelerate Softalk Apple Project Digitization and R&D

Jim Salmons — Sat, 14 Feb 2015 18:32:55 +0000

FactMiners.org and The Softalk Apple Project are very pleased to announce our partnership with the Internet Archive that will dramatically accelerate the digitization, preservation, and public access to an archival-quality digital collection of the full 48-issue run of the Apple edition of Softalk magazine. By dramatically collapsing the pre-research activity of creating the core/source digital archive at The Softalk Apple Project, we will be able to shift considerable resources and move up the schedule to concentrate on the applied R&D initiatives of FactMiners.org's museum informatics mission.

While The Softalk Apple Project is partnering with the Internet Archive as a Contributing Library, the Digitizing Sponsors – meaning the "financial partner" paying for the IA scanning center fees – are project founders, Jim Salmons and Timlynn Babitsky.

"Jim and I are funding this scanning project as one of our 'Pay It Forward' actions in celebration of our 25th wedding anniversary this Valentine's Day, and as a way to kick-off the second year of our post-cancer Bonus Rounds." said Timlynn Babitsky, co-founder and Head Archivist of The Softalk Apple Project, "We're going to turn this project into a super-'serious-fun' experience by taking the magazines to the Midwest Regional Scanning Center to document and participate in the actual scanning process. And we'll be doing this as a stop along our way to DPLAFest, the conclave of folks coming together to celebrate and envision the future of the Digital Public Library of America."

"The acceleration and quality improvements that will result from this partnership with the Internet Archive are certainly significant." said Jim Salmons, co-founder of both projects and tech lead of FactMiners.org, "However, the thing that excites me most is how this partnership clears the way for us to dig into the innovative applied museum informatics research and development agenda we are pursuing at FactMiners. I look forward to detailing our increased R&D activity in the weeks ahead. In the meantime, I know that Timlynn and I are going to have an incredibly fun time going to the Internet Archive Scanning Center to meet Jeff Sharpe and his staff, then on to DPLAFest. Considering what Timlynn and I have been through in these last few years, I am just so happy to even have a 25th wedding anniversary much less have a chance to be working with Timlynn on such exciting Citizen Science and History projects."

For Citizen Science/History projects with a similar need, we encourage you to visit the Internet Archive Scanning Services page, then contact Robert Miller, Global Director of Books and prepare to have a REALLY good technical and distribution partner that will help you achieve your wildest project dreams and then reach for more.

Happy-Healthy Vibes to All, and Happy-Healthy 25th Anniversary to Us (Jim & Timlynn) :D :D

P.S. We promise to "crank it up to 11" for the second year of our Bonus Round! Look out Reaper, not done yet!!!
J&T

Congratulations CIDOC-CRM SIG and the Year Ahead for #cidocCRM Development

Jim Salmons — Tue, 03 Feb 2015 20:08:57 +0000

FactMiners joins the Library, Archive, and Museum (LAM) tech community in congratulating the members of the CIDOC-CRM Special Interest Group for its kick-off to 2015 with release of Version 6.0 of the International Council of Museums (ICOM) Conceptual Reference Model. In addition to the major point update to the model's Reference Document, the CIDOC-CRM SIG also announced a collaboration with the UK-based Collections Trust to create a CIDOC-CRM extension for the Trust's widely-used and highly-regarded SPECTRUM standard for museums' collections management systems.

The CIDOC-CRM (tweeted/hashtagged here and elsewhere as #cidocCRM) serves as a metamodel providing definitions and a formal structure for describing the implicit and explicit concepts and relationships used in cultural heritage documentation. FactMiners is committed to #cidocCRM-driven design for our Open Source social game and exploratory personal and professional "serious fun" research platform.

"Early adopter use cases in the museum community are understandably focused on the descriptive, or ontological, aspects of the CIDOC-CRM. Designing and developing new software systems based on a rigorous metamodel is not the first thing you do with such a resource." said Jim Salmons, tech lead, researcher, and founder of FactMiners and The Softalk Apple Project, "We are very encouraged by publication of the new edition of the Reference Document together with the SPECTRUM announcement. Given the mission and community of The Collections Trust, this is very good news for any museum techies and researchers interested in #cidocCRM-based development projects."

"For our part," Salmons continued, "FactMiners is announcing the launch of our 'Year of the #cidocCRM Full-Graph Deep Dive'. We're asking Kindred Spirits interested in #cidocCRM-driven software design and development to tweet and tag their posts and communications with #cidocCRMdev and #cidocCRMgraph. These hashtags will help focus communication within the museum and broader LAM tech communities specific to these important emerging topics."

In addition to rallying interest in #cidocCRM-driven development within the museum informatics community, FactMiners will increase its outreach activity to engage the Neo4j and broader graph database communities in the exploration and development of a "full graph" implementation of the #cidocCRM.

FactMiners will also engage Neo Technology, the Structr folks, and vendors in the graph visualization community, like KeyLines, to join us in making a contribution to the global cultural preservation movement by working together this year to create a new Neo4j/Structr-based replacement for the www.CIDOC-CRM.org community website. This site is used as a central repository for the #cidocCRM model and its various information resources and news about this ISO standard metamodel. As new developers embrace the #cidocCRM, the community website will be increasingly important. An interactive Neo4j and Structr based site with "live document" visualizations and cross-referencing for the #cidocCRM metamodel will be an ideal way to showcase our web, app, and visualization technologies while making an important contribution to global cultural preservation.

"2015 looks to be the year that the CIDOC-CRM gets a real 'shakedown cruise' as a broader community of software designers and developers begin to explore and use this valuable resource within the cultural heritage community." Salmons said, "And FactMiners is happy to be a part of these exciting developments in the evolution of Internet and its use in the preservation and transmission of our cultural heritage. Keep up the great work CIDOC-CRM SIG! Full speed ahead #cidocCRMdev and #cidocCRMgraph!"

FactMiners and The Softalk Apple Project Are Ashoka/LEGO #Play2Learn Semi-Finalists!

Jim Salmons — Fri, 25 Jul 2014 22:29:34 +0000

We are very excited to announce that both FactMiners and The Softalk Apple Project have been selected as semi-finalist Pacesetter projects in the Ashoka/LEGO #Play2Learn Challenge. Selection as a Pacesetter comes after a thorough screening process in which 636 entries were considered. We are incredibly pleased to be moving on to the next phase of this Challenge.

When Ashoka and the LEGO Foundation posed the question, "Up for the challenge?" on the #Play2Learn Challenge homepage, FactMiners and The Softalk Apple Project jumped right in and got busy. We submitted our inter-related projects -- The Softalk Apple Project and FactMiners. Then the fun began.

As noted in We Want You... to #Play2Learn and Beyond just rolling up your sleeves to make your application a good one is only half of what happens when you participate in an Ashoka Changemakers Challenge. Participating in a Challenge is an opportunity to interact with Kindred Spirits, to meet potential volunteers and advisers, and maybe even to talk with some funding sources.

Ashoka and the LEGO Foundation have initiated the 3-year, multi-pronged Re-imagine Learning #Play2Learn agenda, challenging Changemakers world wide to create innovative projects for learning, in an environment of playful fun.

Participating in the #Play2Learn Challenge has already been an exciting, invigorating process for us. We now look forward to meeting and working with many more of our Pacesetter peers as we move on into round two.

FactMiners.org Proposals ACCEPTED for #MCN2014

Jim Salmons — Wed, 30 Apr 2014 22:26:15 +0000

We are EXTREMELY pleased to report that BOTH our proposals have been accepted for presentation at #MCN2014! Here are our two proposals for presentations at this year's Museum Computer Network conference to be held in Dallas in November. And even better, we are so pleased to report that FactMiners' Jim Salmons has been awarded an Emerging Professional Scholarship to attend #MCN2014! We're going to Dallas! :-)

30-Minute Presentation Proposal

30-Minute Presentation: 30-minute presentations are deep explorations into specific concepts, often illustrated with examples and case studies from the field. The presentations are typically 20 minutes, with an additional 10 minutes for audience questions. 30-minute presentations will usually be thematically grouped together with one or more other presentations to create 90-minute sessions. Choose this option if you have a specific concept you would like to explore in depth.

Title: The Embedded Metamodel Subgraph Design Pattern: Its Use in Crowdsourced Social-Gaming LOD Repository Generation at The Softalk Apple Project

Abstract: This 30-Minute Presentation is a companion to our proposal for a 10-Minute Case Study, "The FactMiners Social Game Ecosystem."

The FactMiners ecosystem is a social gaming community supported by the FactMiners Open Source developers community (www.FactMiners.org). Our "serious fun" playgrounds are the OpenCulture/OpenData digital collections of LAM (Library, Archive, and Museum) hosts of FactMiners Fact Clouds.

FactMiners Fact Clouds are LOD (Linked Open Data) repositories constructed through the crowdsourced gameplay of FactMiners players. LAM hosts interested in adding a Fact Cloud to their visitor and researcher resources use the FactMiners Fact Cloud Wizard to create a domain-specific extension of the CIDOC-CRM Conceptual Reference Model (hereafter 'cidcoCRM'). These Wizard-generated cidocCRM-compliant domain-specific metamodel instances guide players' generation of new 'facts' about the LAM host's collection.

FactMiners domain-specific cidocCRM extensions can be used in the more-typical descriptive realm of folksonomy (social-tagging) and Open Culture visitor-contributed stories and interpretation. But our project is particularly focused on taking the Fact Cloud Wizard's generated metamodel semantics into the "inner space" of textual and image symbolic and/or referential meaning -- i.e., what the words mean and what we 'see' in the image.

In this deep-dive presentation, we'll detail the FactMiner's use of the "embedded metamodel subgraph" design pattern as it is being applied in the Neo4j graph database implementation of the "self-descriptive" Fact Cloud LOD repositories. The case study example detailed will be The Softalk Apple Project, the first FactMiners Fact Cloud under development.

The presentation will include detailed examination of the cidocCRM metamodel as expressed in a Neo4j graph database with particular attention paid to domain-specific extension points and lessons learned while developing this project.

This exploration will be done 'in vivo' through interactive exploration of the metamodel and accompanying 'actual' data through a collection of Cypher queries on the Neo4j graph database. A side effect of attending this presentation will be learning about to the Neo4j graph database and its Cypher query language through a relevant and non-trivial example -- a magazine is a remarkably complex cidocCRM artifact, as will be demonstrated by this presentation.

The Softalk magazine digital archive of the STAP project is a unique historical collection consisting of 48 monthly issues (9,300+ pages), published between September 1980 and August 1984, detailing the Dawn of the Microcomputer Revolution which so significantly shaped the world we live in today. The FactMiners Fact Cloud under development will be a qualitatively new and innovative "lens" on this previously unavailable* resource. (* While collectors have sets of physical issues and various collectors have made scans of select issues available on-line, the STAP project is collaborating with the Stanford Digital Repository for archival storage of the complete 48-issue collection in high-resolution page-wise reference images as well as full-issue PDFs, etc.)

At proposal submission deadline (April 30th), the best representative resources demonstrating the work in process are these:

* The first two parts of a 4-part GraphGist design document submitted to (and won 1st place in its category within) the Neo4j GraphGist Domain Model Winter Challenge:

- Part 1 - The 'Hello World' Introduction to the Embedded Metamodel Subgraph Design Pattern
http://gist.neo4j.org/?8640853

- Part 2 - The 'Self-Descriptive' Neo4j Graph Database: Exploring The Softalk Apple Project's FactMiners Fact Cloud Metamodel
http://gist.neo4j.org/?7817558

* Initial activity importing the cidocCRM Conceptual Reference Model into Neo4j graph database for use as a Reference Model within the Fact Cloud Metamodel Wizard (tool generating cidocCRM-compliant domain-specific metamodel instances):

- A Py2Neo Python utility to parse the CIDOC-CRM official text document into a Neo4j graph database
https://github.com/FactMiners/factminers/tree/master/Documents/Reference...

- Work-In-Process GraphGist, initial activity exploring the cidocCRM in Neo4j
http://jexp.github.io/graphgist/?9904584

* Currently in development but not published (on-line)

- A second-edition of Part 2 of the above GraphGist design document is in development that recasts first edition metamodel as a domain specific extension of the cidocCRM.

- A 'next step' enhancement to the Neo4j reference dataset for the cidocCRM. The current dataset consists of the cidocCRM Class Declarations as nodes and CRM Properties as graph Relationships. The next step is to "node-ify" the cidocCRM Property Declarations with respective Domain and Range (Neo4j graph) relationships to the appropriate Class Declarations. This representation, BTW, is both consistent with the textual presentation of the cidocCRM AND the embedded metamodel subgraph design pattern.

If possible and allowed by the Program Committee reviewers, I will update this proposal abstract with additional information as it becomes available before the committee's review, especially links to the not-yet-on-line resources mentioned above and in active short-term development. If proposal abstract updating is not possible, I will update information on this page: www.FactMiners.org/MCN2014

In the proposed 10-minute companion case study, we'll provide an overview of the FactMiners social network ecosystem and the Open Source web services platform supporting this LAM-focused social gaming community.

Reference Links:
www.FactMiners.org
www.SoftalkApple.com
http://gist.neo4j.org/?7817558
http://www.softalkapple.com/blogs/factminers-more-or-less-folksonomy

Theme 1: OpenData/OpenAuthority
Theme 2: Software/Programming Languages
Theme 3: Curatorial/Scientist/Historian

Presenter Biography: Jim Salmons is Founder and lead designer/developer of the FactMiners ecosystem, and Founder and Project Director of The Softalk Apple Project. He is an independent researcher/developer (AKA "citizen scientist/historian") and former Executive Consultant in the Object Technology Practice of IBM Global Services where he was a lead designer/developer of Smalltalk-based "executable business model" technologies -- work which now informs his vision for the FactMiners ecosystem. For more information: http://sohodojo.biz/about/JimSalmons or https://www.linkedin.com/in/jimsalmons

10-Minute Case Study Proposal

10-Minute Case Study: Case studies are demonstrations of specific projects (whether complete or in-progress). Presenters will have 10 minutes to give an overview of the project, and then will have an opportunity to meet with other attendees for deep-dive discussions. Choose this type if you would like to demonstrate a specific unique project you’re working on.

Title: The FactMiners Social Game Ecosystem

Presenter: Jim Salmons (@Jim_Salmons), Founder and Project Director, FactMiners.org

Abstract: This 10-Minute Case Study is a companion to our proposal for a 30-Minute Presentation, "The Embedded Metamodel Subgraph Design Pattern: Its Use in Crowdsourced Social-Gaming LOD Repository Generation at The Softalk Apple Project."

In the proposed 30-minute companion presentation, we'll detail the FactMiner's use of the "embedded metamodel subgraph" design pattern as it is being applied in the Neo4j graph database implementation of the "self-descriptive" Fact Cloud LOD repositories. The case study example detailed in the 30-minute presentation will be The Softalk Apple Project, the first FactMiners Fact Cloud under development.

In this demonstration we will provide an overview of the social network (people and organizations) that make up the FactMiners social game ecosystem, and we'll briefly explain how the FactMiners technical platform supports this ecosystem.

The emphasis will be on describing the role of the FactMiners.org Open Source developers community. In particular, we'll describe our active "itches needing scratching" areas of activity. Our goal in presenting at #MCN2014 is to identify Kindred Spirits interested in joining the FactMiners.org developers community or who want to collaborate with us (or point us to helpful resources) as we pursue our mission of Serious Fun for Everyone.

Reference Links:
www.FactMiners.org
www.SoftalkApple.com
http://gist.neo4j.org/?7817558
http://www.softalkapple.com/blogs/factminers-more-or-less-folksonomy

Theme 1: OpenData/OpenAuthority
Theme 2: Software/Programming Languages
Theme 3: Curatorial/Scientist/Historian

FactMiners Fact Clouds to be CIDOC-CRM Compliant

Jim Salmons — Sun, 23 Feb 2014 01:32:47 +0000

FactMiners.org is pleased to announce adoption of the Conceptual Reference Model (CRM) of the International Committee for Documentation (CIDOC) of the International Council of Museums (ICOM). The CIDOC-CRM is an 'ontology' for cultural heritage information, in other words it describes in a formal language the concepts and relations relevant to the documentation of cultural heritage. This ISO standard will be used as the reference model for the development of the FactMiners Fact Cloud metamodel that will logically organize the 'facts' to be 'mined' out of our initial project — the digital archive of Softalk magazine — respecting both the elements of its editorial content and the complex document structure of a magazine.

It is truly exciting to be able to 'stand on the shoulders of giants' with respect to formalizing the candidate elements and overall logical organization of the facts that we will be capturing in the FactMiners Fact Cloud describing the content of Softalk magazine. This reference standard will dramatically accelerate our design and implementation of the FactMiners Fact Cloud Wizard component within the core FactMiners Open Source development platform.

A relatively small group of dedicated data scientists and museum informatics professionals have spent nearly twenty years working out an essential set of elements and their logical organization. This means we can get right to work developing our FactMiners Fact Cloud companion of the Softalk archive as a domain-specific extension of the CIDOC-CRM reference model. Doing this, we guarantee Semantic Web and OpenData accessibility which is essential to our project mission.

Our growing collaboration with the www.Structr.org team will benefit greatly from this reference model as we work together to envision and build the FactMiners platform on the Neo4j-powered Structr CMS/web-services platform. Our first efforts will focus on the FactMiners Fact Cloud Wizard described in Part 2 of this Neo4j GraphGist.

Neo4j GraphGist Design Docs On-line

Jim Salmons — Sun, 02 Feb 2014 23:02:37 +0000

UPDATE: I'm very pleased to note that this 2-part GraphGist won First Place in the Other category of the Neo4j GraphGist Winter Domain Modeling Challenge! :-)

I have been very quiet during the year-end holiday and throughout January. I have been busy in the 'deep weeds' of moving the FactMiners social-game ecosystem forward. Quick summary... The FactMiners game is the means we will use to create an incredible 'Fact Cloud' of all the information in all 48 issues of Softalk magazine. It is an ambitious mission, but one that is guaranteed to be "serious fun"– especially when we create a crowdsource game technology and community (that's the ecosystem scope of this).

I have posted two of four parts as an entry in the Neo4j GraphGist Winter Challenge. This multi-part GraphGist presents the "embedded metamodel subgraph" design pattern underlying the FactMiners ecosystem: