Developer Guide

Contents:

• Introduction
  • Intended Audience
  • Core Technologies
  • Issue Tracker
  • Related Guides
  • Related Projects
• Windows Development
  • Running Dataverse in Vagrant
    • Install Vagrant
    • Install VirtualBox
    • Reboot
    • Install Git
    • Configure Git to use Unix Line Endings
    • Clone Git Repo
    • vagrant up
  • Running Dataverse in Minishift
    • Install VirtualBox
    • Install Git
    • Install Minishift
    • Clone Git Repo
    • Start Minishift VM and Run Dataverse
  • Improving Windows Support
    • Windows Subsystem for Linux
    • Discussion and Feedback
• Tips
  • Iterating on Code and Redeploying
    • Undeploy the war File from the install Script
    • Add Glassfish 4.1.2 as a Server in Netbeans
    • Ensure that Dataverse Will Be Deployed to Glassfish 4.1.2
    • Make a Small Change to the Code
    • Confirm the Change Was Deployed
  • Netbeans Connector Chrome Extension
  • Database Schema Exploration
    • pgAdmin
    • SchemaSpy
  • Deploying With asadmin
  • Running the Dataverse install Script in Non-Interactive Mode
  • Preventing Glassfish from Phoning Home
  • Solr
  • Git
    • Set Up SSH Keys
    • Git on Mac
• Troubleshooting
  • context-root in glassfish-web.xml Munged by Netbeans
  • Configuring / Troubleshooting Mail Host
  • DataCite
• Version Control
  • Where to Find the Dataverse Code
  • Branching Strategy
    • Goals
    • Branches
      • The “master” Branch
      • The “develop” Branch
      • Feature Branches
  • How to Make a Pull Request
    • Find or Create a GitHub Issue
    • Create a New Branch off the develop Branch
    • Commit Your Change to Your New Branch
    • Push Your Branch to GitHub
    • Make a Pull Request
  • How to Resolve Conflicts in Your Pull Request
  • Adding Commits to a Pull Request from a Fork
• SQL Migration
  • Location of SQL Upgrade Scripts
  • How to Determine if You Need to Create a SQL migration
  • How to Create a SQL
  • Troubleshooting
    • Renaming SQL
• Testing
  • The Health of a Codebase
  • Testing in Depth
  • Locale
  • Unit Tests
    • Unit Test Automation Overview
    • Writing Unit Tests with JUnit
      • Parameterized Tests and JUnit Theories
      • Observing Changes to Code Coverage
      • Testing Commands
      • Running Non-Essential (Excluded) Unit Tests
  • Integration Tests
    • Database
    • Docker
• Documentation
  • Quick Fix
  • Other Changes (Sphinx)
    • Installing Sphinx
    • Using Sphinx
  • Table of Contents
  • Images
    • GraphViz based images
  • Versions
• Dependency Management
  • Terms
  • Direct dependencies
  • Transitive dependencies
    • Managing transitive dependencies in pom.xml
  • Helpful tools
  • Repositories
• Debugging
  • Logging
• Coding Style
  • Java
    • Formatting Code
      • Format Code You Changed with Netbeans
      • Checking Your Formatting With Checkstyle
    • Logging
  • Bash
    • Formatting Code
      • Tabs vs. Spaces
• Deployment
  • Deploying Dataverse to Amazon Web Services (AWS)
    • Install AWS CLI
      • Troubleshooting “aws: command not found”
    • Configure AWS CLI
    • Configure Ansible File (Optional)
    • Download and Run the “Create Instance” Script
    • Caveats
• Docker, Kubernetes, and OpenShift
  • OpenShift
    • Install Minishift
    • Start Minishift
    • Make the OpenShift Client Binary (oc) Executable
    • Log in to Minishift from the Command Line
    • Create a Minishift Project
    • Create a Dataverse App within the Minishift Project
    • Log into Minishift and Visit Dataverse in your Browser
    • Troubleshooting
      • Check Status of Dataverse Deployment to Minishift
      • Review Logs of Dataverse Deployment to Minishift
      • Get a Shell (ssh/rsh) on Containers Deployed to Minishift
    • Cleaning up
    • Making Changes
      • Making Changes to Docker Images
      • Making Changes to the OpenShift Config
      • Making Changes to the PostgreSQL Database from the Glassfish Pod
    • Scaling Dataverse by Increasing Replicas in a StatefulSet
    • Configuring Persistent Volumes and Solr master node recovery
    • Running Containers to Run as Root in Minishift
    • Minishift Resources
  • Docker
    • Installing Docker
    • All In One Docker Images for Testing
    • Future production use on Minishift/OpenShift/Kubernetes
      • Known Issues with Dataverse Images on Docker Hub
• Making Releases
  • Create the release GitHub issue and branch
    • 1. Bump Version Numbers
    • 2. Check in the Bumping Version Numbers Changes…
  • Merge “develop” into “master”
  • Write Release Notes
  • Make Artifacts Available for Download
  • Publish Release
• Tools
  • Netbeans Connector Chrome Extension
  • pgAdmin
  • Maven
  • Vagrant
  • PlantUML
  • Eclipse Memory Analyzer Tool (MAT)
  • PageKite
  • MSV
  • FontCustom
  • SonarQube
  • Infer
  • lsof
• Universal Numerical Fingerprint (UNF)
  • UNF Version 3
  • UNF Version 5
  • UNF Version 6
    • I. UNF of a Data Vector
    • II. Combining multiple UNFs to create UNFs of higher-level objects.
    • Footnotes:
• Shibboleth and OAuth
  • Shibboleth and OAuth
• Geospatial Data
  • Geoconnect
  • How Dataverse Ingests Shapefiles
    • Ingest
    • Example
  • WorldMap JoinTargets + API Endpoint
    • How Geoconnect Uses Join Target Information
    • Retrieving Join Target Information from WorldMap API
    • Saving Join Target Information to Geoconnect Database
    • Setting Up WorldMap Test Data
• SELinux
  • Introduction
  • Development Environment
  • Recreating the shibboleth.te File
    • Ensure that SELinux is Enforcing
    • Removing the Existing shibboleth.te Rules
    • Exercising SELinux denials
    • Stub out the new shibboleth.te file
    • Iteratively Use audit2allow to Add Rules and Test Your Change
• Big Data Support
  • Data Capture Module (DCM)
    • Install a DCM
    • Downloading rsync scripts via Dataverse API
    • How a DCM reports checksum success or failure to Dataverse
    • Steps to set up a DCM mock for Development
    • Troubleshooting
    • Steps to set up a DCM via Docker for Development
      • Docker Image Set-up
      • Optional steps for setting up the S3 Docker DCM Variant
      • Using the DCM Docker Containers
      • Additional DCM docker development tips
  • Repository Storage Abstraction Layer (RSAL)
    • Configuring the RSAL Mock
    • Configuring download via rsync
• Workflows
  • Introduction
    • Administration
    • Available Steps
      • log
      • pause
      • http/sr
      • archiver