Federated Approach to Code Sharing 

Jason Duley

OCIO T&I Division/ARC

jason.duley@nasa.gov

4/15/2016

Goals

• Goals of a successful Code Sharing System:

  • Re-use, Re-use, Re-use

  • Provides Localized Access Control

  • Empowers Localized DevOps

  • Heterogenous Architecture

  • Reduction of Copies/Orphan Branches

  • Automates

    • Software Release Process

    • Meta-data collection

How

• Approaches: Centralized vs Federated

  • Centralized vs Federated Code Sharing architecture is not an "OR" but an "AND" and requires concepts from both paradigms

  • Need to overcome short-comings of pure Centralized approach and meet goals developers demand using a minimally intrusive and loosely-coupled Federated Approach

Culture meets reality

• NASA Missions/Projects/Programs are highly distributed, highly diverse and require IT systems to match this reality

• These parties require localized control to match their requirements based on mission type, funding, CM versioning system of choice, level of security, DevOps customization, etc.

• The agency needs a way to integrate these disparate systems that; give teams control to work effectively and provide the ability to have insight into all of NASA's software efforts

Federated Approach

• Big Idea
  • Provide some "centralized" systems such as developer.nasa.gov, GitHub.com/NASA, ipas-gitlab, etc. for teams that do not have the IT resources to setup their own
  • Embrace teams that leverage existing center, workgroup and mission-based CM systems to participate
  • Federate all of these

Benefits

• Idea
  • Do for Heterogeneous distributed Code Repositories what Google did for Web Servers - provide a unified interface to find stuff in one place
  • NASA users will be able to find code on geographically dispersed systems without regard for actual host location
• Why
  • Allows localized control with agency-wide visibility

Progress

• Initial prototype developed and running in the Ames datacenter on small dev VM
• Received funding to build Phase 1 of system (expand prototype)
• Work for Phase 1
  • Catalog existing nasa.gov domain code versioning instances across the agency 
  • Connect to participating repositories in each of those instances to form federation of repositories
  • Enable full-text search and distributed code browsing from a single web application
  • Apply metadata to projects and enable facet-search on projects to boost re-use and discoverability

Federated Code Sharing

Federated Code Sharing

Federated Code Sharing

Meta data

• Projects define meta data in JSON serialized descriptor files that live with the project

• Software/Code owners can define their own facet values which are dynamically displayed in the federation browser at runtime

• Architecture can support extensive descriptor vocabularies to increase specificity of metadata

Backup

Process

Code Sharing

• Open Source Software publication

  • 180+ Released Open Source projects

  • Community-driven via Open Software Catalog

  • Search Projects, SRA Contact Info, Process Information

http://code.nasa.gov

Code Sharing

• Internal NASA Code Sharing

• Heterogenous Federated Code Sharing System Prototype

• Repositories (GIT / SVN / Hg / CVS / Custom)

• Distributed Repository Browsing

• Full-text Search of Source Code

• Faceted Search of Projects

• Leverage Data Standards to model software projects (Distributed Software Registry)

Code Sharing

Distributed Software Registry

• Problem: Need a way to describe Software Projects using a standard controlled vocabulary 
• How: Leverage Data On A Project (DOAP) RDF vocabulary to model software assets
• Challenge: Need a simplified way to allow software owners create/manage descriptions
• Solution: Create a framework to allow authoring of project descriptions which are easy for developers to use, but create metadata descriptions with enough specificity