What I Know About Vagrant So Far

Date

Sep 24, 2015

Attendees

• @pketienne

• @Mark Bussey - Digital Curation Experts

• @Steven Ng - Temple University

• @aliciac - Digital Curation Experts

• Many Others

Goals

• Discuss Vagrant and what it offers for streamlining development or devops processes.

• Discuss other orchestration tools and their potential use for automating the workload of devops.

Summary:

• Conversation is the order of the day.

• What does mark know? what do we know?

• a lot of people use vagrant (~15/23) for virtualization of some kind.

• DCE works with many different institutions so the abstraction of virtualization processes is important.

• The stack is very complicated. Virtualization is good for testing / scripting edits, changes so it doesn't need to be done in production.

• DCE is primarily a Mac shop, so is the majority of the community (by vote of laptop brands at least), so non-apple users contributions to the community is important for other non-apple users' adoption of hydra.

• Institutions and organizations at hydra use all sorts of virtualization environments, Amazon Web Services, Google Compute Engine, OpenStack, WebSphere, etc.

• Mark and Alicia have built a lot of VMs on AWS and are working toward a simpler process for building VMs that can deploy to AWS and may other end targets.

• InfoGraphic - Sections / Layers: Platform, Bootstrap, Hydra Stack, Application - the four areas of deployment.

• Concerning VMs for Hydra Camp: need to support all sorts of environments. Problems crop up all over trying to get environments up and running for teaching initiatives.

• Mark showcases his development:

  • Iteration 1: github.com/mark-dce/camper

  • Iteration 2: Mark learned how to use kickstarter files to build their own boxes from scratch (bootstrap layer).

    • Talk of ssh keys for generic users.

    • Included installation steps - virtual box guest additions, etc

• Configuration manage all the things!

• The process of using "orchestration management tools" (in this case, vagrant, ansible, and AWS) took DCE's workload from being measured in days (3-4) to being measured in minutes (10-20?).

• Are the bootstrapped boxes published (somewhere) after the work is complete?

  • Mostly published on thumb drives due to the purpose of Hydra Camp and related issue of network congestion.

  • But they can be published somewhere.

• DCE uses different versions of files for different targets: local boxes, AWS, VSphere, etc.

• Kieran's (my) spiel about packer.io - roughly that Packer can take the role of managing those different versions of files for different targets. Works with all major orchestration tools.

• Connecting to a VM as the vagrant user - problematic when attempting to commit development work.

  • A behavioral solution: use "vagrant" only as a "build" user (should be rare) while manually or as a part of the orchestration tool process adding an ssh key (or keys) for a user (users) who will be committing code from that environment.

  • A possible technological solution:  use https rather than ssh.

• Having these orchestration tools seems incredibly useful to the present members of the community.

• Brief conversation about docker - put docker images on "the registry".

  • like ansible galaxy

  • like puppet modules

  • like hashicorp images

• Is this (the pursuit of researching and implementing orchestration tools) an initiative within the community?

• Where should information about his initiative live?

• Possibility of an interest group? Alicia, Kieran (myself), Mark, (someone else behind me I couldn't see).

• Hydra-in-a-Box community would be very interested in the results of this conversation (id est, Joseph Atzberger)

• What about huge files and using vagrant?

  • No immediate solution.

• Next Steps

  • Interest group

  • Documentation for the meeting

  • Be gentle about pull requests.