Scott's Weblog The weblog of an IT pro specializing in cloud computing, virtualization, and networking, all with an open source view

Experimenting with Azure

I’ve been experimenting with Microsoft Azure recently, and I thought it might be useful to share a quick post on using some of my favorite tools with Azure. I’ve found it useful to try to leverage existing tools whenever I can, and so as I’ve been experimenting with Azure I’ve been leveraging familiar tools like Docker Machine and Vagrant.

The information here isn’t revolutionary or unique, but hopefully it will still be useful to others, even if only as a “quick reference”-type of post.

Launching an Instance on Azure Using Docker Machine

To launch an instance on Azure and provision it with Docker using docker-machine:

docker-machine create -d azure \
--azure-subscription-id $(az account show --query "id" -o tsv) \
--azure-ssh-user azureuser \
--azure-size "Standard_B1ms" azure-test

The first time you run this you’ll probably need to allow Docker Machine access to your Azure subscription (you’ll get prompted to log in via a browser and allow access). This will create a service principal that is visible via az ad sp list. Note that you may be prompted for authentication for future uses, although it will re-use the existing service principal once it is created.

Launching an Instance Using the Azure Provider for Vagrant

See this page for complete details on using the Azure provider for Vagrant. Basically, it boils down to these four steps:

  1. Install the Azure provider using vagrant plugin install vagrant-azure.
  2. Add a “dummy” box (similar to how you use Vagrant with AWS; see this post).
  3. Set up an Azure service principal for Vagrant to use to connect to Azure.
  4. Run vagrant up and you’re off to the races.

A more detailed post on using Vagrant with Azure is available here; it provides a bit more information on the above steps.

Launching an Instance using the Azure CLI

OK, maybe the Azure CLI isn’t exactly an “existing tool,” but given my affinity for CLI-based tools I think it’s probably reasonable to include it here. To launch an instance using the Azure CLI, it would look something like this:

az vm create -n vm-name -g group-name --image UbuntuLTS --size Standard_B1ms --no-wait

Of course, this assumes a pre-existing resource group. More details are available here.

If you need to install the Azure CLI, see here or here for some additional information.

Happy experimenting!

Issue with VMware-Formatted Cumulus VX Vagrant Box

I recently had a need to revisit the use of Cumulus VX (the Cumulus Networks virtual appliance running Cumulus Linux) in a Vagrant environment, and I wanted to be sure to test what I was doing on multiple virtualization platforms. Via Vagrant Cloud, Cumulus distributes VirtualBox and Libvirt versions of Cumulus VX, and there is a slightly older version that also provides a VMware-formatted box. Unfortunately, there’s a simple error in the VMware-formatted box that prevents it from working. Here’s the fix.

The latest version (as of this writing) of Cumulus VX was 3.5.0, and for this version both VirtualBox-formatted and Libvirt-formatted boxes are provided. For a VMware-formatted box, the latest version is 3.2.0, which you can install with this command:

vagrant box add CumulusCommunity/cumulus-vx --box-version 3.2.0

When this Vagrant box is installed using the above command, what actually happens is something like this (at a high level):

  1. The *.box file for the specific box, platform, and version is downloaded. This .box file is nothing more than a TAR archive with specific files included (see here for more details).

  2. The *.box file is expanded into the ~/.vagrant.d/boxes directory on your system. A directory tree is built that helps Vagrant support multiple versions of the same box along with multiple formats of the same box (for example, having version 3.2.0 of a VMware-formatted box alongside version 3.5.0 of a VirtualBox-formatted box on the same system).

In this case, when you install version 3.2.0 of the VMware-formatted Cumulus VX Vagrant box, you’ll end up with a set of files found in ~/.vagrant.d/boxes/CumulusCommunity-VAGRANTSLASH-cumulus-vx/3.2.0/vmware_desktop. In this directory, you’ll find all the files that would describe a VM to a product like VMware Fusion or VMware Workstation: the VMX file, one or more VMDK files, etc.

What you’ll also find for this particular box is something you don’t want: a lock file, in the form of a directory named cumulus-linux-3.2.0.vmx.lck. This lock file is normally created by a VMware desktop virtualization product to indicate that the VM is running and therefore the files are locked and can’t be accessed. Unfortunately, the presence of this directory means that the Vagrant box will not work.

If you try to run vagrant up on a Vagrant enviroment with this box, you’ll get an error indicating the files are locked, and the vagrant up command will fail.

So how does one fix this?

Simple: just delete the cumulus-linux-3.2.0.vmx.lck directory and its contents.

Once you’ve deleted that file, then using vagrant up to instantiate a Vagrant environment based on this box will work as expected.

(Side note: If you are planning to use version 3.2.0 of the VMware-formatted Cumulus VX box, there’s one additional oddity. When you use vagrant box add as outlined above to download and install the box, you’ll be prompted with a set of options for which provider to use. Be sure to use option 4—the one labeled “vmware_desktop”—and not option 5, labeled “vmware_fusion”. The latter reports an error after downloading the box and the command fails.)

Hopefully Cumulus Networks will release an updated version of the Cumulus VX Vagrant box for VMware products that addresses these issues.

Technology Short Take 92

Welcome to Technology Short Take 92, the first Technology Short Take of 2018. This one was supposed to be the last Tech Short Take of 2017, but I didn’t get it published in time (I decided to spend time with my family instead—some things are just more important). In any case, hopefully the delay of one additional week hasn’t caused any undue stress—let’s jump right in!

Networking

  • Lindsay Hill walks through using Telegraf, InfluxDB, and Grafana to monitor network statistics.
  • Via Ivan Pepelnjak, I found this article by Diane Patton at Cumulus Networks talking about container network designs. The article is a bit heavy on pushing the Host Pack (a Cumulus thing), but otherwise provides a good overview of several different possible container network designs, along with some of the criteria that might lead to each design.
  • Erik Hinderer takes a stab (based on his field experience) at estimating how long it takes to upgrade VMware NSX. Erik’s figures are just estimates, of course; actual values will be determined based on each customer’s specific environment.
  • This post is a bit older, but covers a challenge faced by cloud-native darling Netflix—how does one, exactly, identify which application used which IP address at a given point in time? When you’re operating at the scale at which Netflix operates, this is no trivial feat.

Servers/Hardware

Security

  • The CPU architecture flaw involving speculative execution has been garnering a great deal of attention (see here, here, here, and here). Also, here’s Google Project Zero’s write-up (along with a support FAQ from Google on mitigation). There’s lots more coverage, obviously, but this should be enough to get you started.

Cloud Computing/Cloud Management

  • Kevin Carter has a detailed write-up on efforts around leveraging systemd-nspawn for deploying OpenStack via OpenStack Ansible. systemd-nspawn is an interesting technology I’ve been watching since early this year, and it will be cool (in my opinion) to see a project using it in this fashion.
  • The vSphere provider for Terraform (did you know there was one?) recently hit 1.0, and HashiCorp has a blog post (re-)introducing the provider. I thought I also saw a VMware blog post on the provider as well, but couldn’t find any link (guess I was mistaken).
  • Oh, and speaking of Terraform: check out this post on the release of Terraform 0.11.
  • Tim Nolet reviews some differences between Azure Container Instances and AWS Fargate (recently announced at AWS re:Invent 2017). Tim’s review of each of the offerings is pretty balanced (thanks for that), and I’d recommend reading this post to get a better idea of how each of them work.
  • Jorge Salamero Sanz (on behalf of Sysdig) provides a similar comparison, this time looking at ECS, Fargate, and EKS. Jorge’s explanation of Fargate as “managed ECS/EKS instances” is probably the most useful explanation of Fargate I’ve seen so far.
  • Michael Gasch digs relatively deep to address the question of how Kubernetes reconciles allocatable resources and requested resources in order to satisfy QoS. Good information here, in my opinion. Thanks Michael!
  • Running distributed systems such as etcd, Kubernetes, Linkerd, etc., to support applications means making a conscious decision to embrace a certain level of complexity in exchange for the benefits these systems offer. Read this post-mortem on an outage to gain a better idea of some of the challenges this additional complexity might present when it comes to troubleshooting.
  • Tim Hinrichs provides some details on Rego, the policy language behind the Open Policy Agent project.
  • Paul Czarkowski walks you through creating your first Helm chart.

Operating Systems/Applications

  • I came across this mention of Mitogen, a project whose goal—as described by the creator—is to “make it childsplay [sic] to run Python code on remote machines”.
  • From the “interesting-but-not-practicallly-useful” department, Nick Janetakis shows how to use Docker to run a PDP-11 simulator. The magic here, in my opinion, is in the simulator (not in Docker), but it’s still an interesting look at how one might use Docker.
  • Also from Nick, here’s an attempt to the answer the question, “Do I learn Docker Swarm or Kubernetes?”
  • I debated on adding this link because I wasn’t sure how useful it might be to readers, but decided to include it anyway. Apache Guacamole describes itself as “a clientless remote desktop gateway” supporting standard protocols like SSH, VNC, and RDP.
  • Tamás Török has a quite lengthy post on transforming your system into microservices. It’s nice to see some of the challenges—which aren’t all technical—mentioned as well, as sometimes today’s tech writers only seem to see microservices through rose-colored glasses.
  • This is an awesome collection of patched fonts.
  • OpenSSH on Windows—what a time to be alive! It almost makes me want to add a Windows 10 machine to my collection…
  • I enjoyed this developer-centric comparison of Kubernetes and Pivotal Cloud Foundry.

Storage

  • Tony Bourke has a two-part series on ZFS and Linux and encryption (part 1, part 2).

Virtualization

Career/Soft Skills

  • Although targeted at “creatives,” I think there are some tips and ideas in this post that are equally applicable to IT professionals.

That’s it for this time around. Look for the next Technology Short Take in a couple of weeks, where I’ll have another curated collection of links and articles for you. Until then, enjoy!

Looking Back: 2017 Project Report Card

As has become my custom for the past several years, I wanted to take a look at how well I fared on my 2017 project list. Normally I’d publish this before the end of 2017, but during this past holiday season I decided to more fully “unplug” and focus on the truly important things in life (like my family). So, here’s a look back at my 2017 projects and a report card on my progress (or lack thereof, in some cases).

For reference, here’s the list of projects I set out for myself in 2017:

  1. Finish the network automation book.
  2. Launch an open source book project.
  3. Produce some video content.
  4. Get the Full Stack Journey podcast back on track.
  5. Complete a “wildcard project.”

So, how did I do with each of these projects?

  1. Finish the network automation book: I’m happy to report that all the content for the network automation book I’ve been writing with Jason Edelman and Matt Oswalt is done, and the book is currently in production (and should be available to order from O’Reilly very soon). I had hoped to get the content done in time for the book to be available for order before the end of 2017, so I’m marking myself down just a bit on this one. Grade: B

  2. Launch an open source book project: I launched The Open vSwitch Cookbook in late February, and then canceled the project in late March. Why? Basically, it boils down to my effort detracting from the effort to include high-quality documentation with Open vSwitch itself, and I decided it was better to support the efforts of the OVS project than put forth my own (competitive) project. Because I learned something from this project—how to better align my own efforts with the efforts of open source projects I want to support—I don’t consider this to be a total failure. Grade: D

  3. Produce some video content: I intended to start producing some video content, such as demos of a project or a video “how-to” for a certain technology. Unfortunately, I simply didn’t make it. Grade: F

  4. Get the Full Stack Journey podcast back on track: I was aiming for nine episodes in 2017, and I managed to publish six. More importantly, though, was that I was able to join forces with Packet Pushers, giving me a support network to make the podcast even better (in the long run). Grade: B

  5. Complete a “wildcard” project: I’ve had this on my list for the last few years, as a way of trying to account for unseen changes in the industry that may pull me in a direction different than what I had anticipated. I did lots of interesting things this past year, but the only “project” that stands out was the migration of this web site from Jekyll on GitHub Pages to Hugo on Amazon S3/CloudFront (more information is available here). This migration was very smooth and has—I think—resulted in a better site with better performance. It’s also pushed me in some new directions, which I think is a good thing. Grade: A

Overall, my progress was reasonable—not stellar, but not awful. (That’s an improvement over last year, at least!) Over the next few weeks, I’ll be evaluating the projects I want to tackle in 2018. Once I have that list ready to share, I’ll publish it here as I have in the past.

Have some feedback for me? Feel free to hit me up on Twitter, or drop me an email (my address is here on the site). Thanks!

Installing XMind 8 on Fedora 27

XMind is a well-known cross-platform mind mapping application. Installing the latest version of XMind (version 8) on Linux is, unfortunately, more complicated than it should be. In this post, I’ll show how to get XMind 8 running on Fedora 27.

So why is installing XMind more complicated than it should be? For reasons unknown, the makers of XMind stopped using well-known Linux package mechanisms with this version of the software, providing only a ZIP archive to download and extract. (Previous versions at least provided a Debian package.) While the ZIP archive includes a very simplistic “setup script”, the script does nothing more than install a few packages and install some fonts, and was written expressly for Debian-based systems. If you extract the archive and place the files outside of your home directory (as would be typical for installing an application on most desktop Linux distributions), you’ll run into problems with permissions. Finally, the application itself is extraordinarily brittle with regards to file locations and such; it’s easy to break it by simply moving the wrong file.

Through some research and some trial-and-error, I finally arrived at a configuration for XMind 8 on Fedora 27 that satisfies a couple criteria:

  1. The application should reside outside the user’s home directory in a location that is typical for third-party applications (for example, in the /opt directory).

  2. All user-specific directories and information would reside in the user’s home directory so as to eliminate the need for overly-permissive file/group permissions.

Here are the steps you can follow to get XMind 8 installed on Fedora 27 in a way that satisfies these criteria.

First, you’ll need to install the “java-1.8.0-openjdk” package using dnf install. XMind has a few different prerequisite packages (lame, webkitgtk, and glibc), but in my tests on Fedora 27 system this was the only package that wasn’t already installed. Note that if you’re trying to replicate these instructions on a different Linux distribution, this step is where you’ll need to locate distribution-specific package names (most of the rest of the steps are applicable to any Linux distribution).

Next, download the XMind 8 ZIP archive, and extract it into an xmind directory:

unzip xmind-8-update6-linux.zip -d xmind

For simplicity’s sake, I chose to work within my own ~/Downloads directory, but you should be able to work from within any directory where you have write permissions.

Third, create a user-specific area for XMind to store information:

mkdir -p ~/.config/xmind/workspace

I chose the ~/.config directory since it was already present and utilized for application-specific information. You can use a different directory, but the rest of the instructions will assume this path was used.

Fourth, go ahead and remove the 32-bit version of the XMind executable; it’s pretty likely you won’t need it:

rm -rf xmind/XMind_i386

Fifth, copy over two directories into the user-specific area you created earlier:

cp -a xmind/XMind_amd64/configuration ~/.config/xmind/
cp -a xmind/XMind_amd64/p2 ~/.config/xmind/

Based on my testing, this step should be the only step needed to make XMind work for additional users on your Fedora system (i.e., you’ll want to run this step for other users on the system as well in order for them to be able to run XMind).

Next, you’ll need to update XMind.ini to tell XMind the new user-specific locations. Edit this file (found in the XMind_amd64 subdirectory), and make the following changes:

  • On line 2, change ./configuration to @user.home/.config/xmind/configuration (the @user.home refers to the user’s home directory; note that you can’t use the tilde shortcut here as it won’t work)
  • On line 4, change ../workspace to @user.home/.config/xmind/workspace

You’re almost done! In the fonts subdirectory of the xmind directory in which you stored the extracted files, you’ll find some fonts that are distributed with XMind. Install these on your system (copy them to ~/.local/share/fonts or /usr/share/fonts, as you desire), and then remove the fonts subdirectory. In my particular case, some of the fonts—like the Roboto family—were already installed.

Finally, move the xmind directory to its final location and lock down the permissions:

sudo mv xmind /opt/
sudo chown -R root:root /opt/xmind

The last and final step is to create a desktop entry file so that XMind is accessible via the launcher. Here’s a sample file:

[Desktop Entry]
Comment=Create and share mind maps
Exec=/opt/xmind/XMind_amd64/XMind %F
Path=/opt/xmind/XMind_amd64
Name=XMind
Terminal=false
Type=Application
Categories=Office;Productivity
Icon=xmind

This desktop file can go into /usr/share/applications or ~/.local/share/applications, though—for reasons I’ll share shortly—the latter may be a better choice.

That’s it! You should be able to launch XMind from the GNOME Activities screen.

Unfortunately, there are some caveats and limitations:

  • XMind apparently stores its application icon buried deep in the ~/.config/xmind/configuration directory structure. This is why you may prefer to use ~/.local/share/applications for the desktop file. If you do use this location, you’ll need to perform this step for other users of the system as well. (The icon=xmind line in the sample desktop file above won’t actually work.)
  • This manual installation doesn’t create a MIME type for XMind documents so that you can open an XMind document from within the Nautilus file manager. (The simplistic shell script supplied in the XMind download doesn’t either.) I’ve done a bit of work on this, but haven’t come to a workable solution yet.

So there you have it: how to get XMind 8 installed and running in some reasonable fashion on Fedora 27. If you have questions, comments, or corrections, feel free to hit me up on Twitter.

Recent Posts

Installing the VMware Horizon Client on Fedora 27

In this post, I’ll outline the steps necessary to install the VMware Horizon client for Linux on Fedora 27. Although VMware provides an “install bundle,” the bundle does not, unfortunately, address any of the prerequisites that are necessary in order for the Horizon client to work. Fortunately, some other folks shared their experiences, and building on their knowledge I was able to make it work. I hope that this post will, in turn, help others who may find themselves in the same situation.

Read more...

Using Vagrant with Azure

In this post, I’ll describe how to use Vagrant with Azure. You can consider this article an extension of some of my earlier Vagrant articles; namely, the posts on using Vagrant with AWS and using Vagrant with OpenStack. The theme across all these posts is examining how one might use Vagrant to simplify/streamline the consumption of resources from a provider using the familiar Vagrant workflow.

Read more...

Technology Short Take 91

Welcome to Technology Short Take 91! It’s been a bit longer than usual since the last Tech Short Take (partly due to the US Thanksgiving holiday, partly due to vacation time, and partly due to business travel), so apologies for that. Still, there’s a great collection of links and articles here for you, so dig in and enjoy. Networking Amanpreet Singh has a two-part series on Kubernetes networking (part 1, part 2).Read more...

Installing the Azure CLI on Fedora 27

This post is a follow-up to a post from earlier this year on manually installing the Azure CLI on Fedora 25. I encourage you to refer back to that post for a bit of background. I’m writing this post because the procedure for manually installing the Azure CLI on Fedora 27 is slightly different than the procedure for Fedora 25.

Read more...

Using Vagrant with Libvirt on Fedora 27

In this post, I’m going to show you how to use Vagrant with Libvirt via the vagrant-libvirt provider when running on Fedora 27. Both Vagrant and Libvirt are topics I’ve covered more than a few times here on this site, but this is the first time I’ve discussed combining the two projects.

Read more...

AWS re:Invent 2017 Keynote with Andy Jassy

This is a liveblog of the re:Invent 2017 keynote with Andy Jassy, taking place on Wednesday at the Venetian. As fully expected given the long queues and massive crowds, even arriving an hour early to the keynote isn’t soon enough; there’s already a huge crowd gathered to make it into the venue. Fortunately, I did make it in and scored a reasonable seat from which to write this liveblog.

Read more...

Liveblog: Deep Dive on Amazon Elastic File System

This is a liveblog of the AWS re:Invent 2017 session titled “Deep Dive on Amazon Elastic File System (EFS).” The presenters are Edward Naim and Darryl Osborne, both with AWS. This is my last session of day 2 of re:Invent; thus far, most of my time has been spent in hands-on workshops with only a few breakout sessions today. EFS is a topic I’ve watched, but haven’t had time to really dig into, so I’m looking forward to this session.

Read more...

Liveblog: IPv6 in the Cloud - Protocol and Service Overview

This is a liveblog of an AWS re:Invent 2017 breakout session titled “IPv6 in the Cloud: Protocol and Service Overview.” The presenter’s name is Alan Halachmi, who is a Senior Manager of Solutions Architecture at AWS. As with so many of the other breakout sessions and workshops here at re:Invent this year, the queues to get into the session are long and it’s expected that the session will be completely full.

Read more...

A Sample Makefile for Creating Blog Articles

In October of this year, I published a blog post talking about a sample Makefile for publishing blog articles. That post focused on the use of make and a Makefile for automating the process of a publishing a blog post. This post is a companion to that post, and focuses on the use of a Makefile for automating the creation of blog posts.

Read more...

Installing MultiMarkdown 6 on Fedora 27

Long-time readers are probably aware that I’m a big fan of Markdown. Specifically, I prefer the MultiMarkdown variant that adds some additional extensions beyond “standard” Markdown. As such, I’ve long used Fletcher Penny’s MultiMarkdown processor (the latest version, version 6, is available on GitHub). While Fletcher offers binary builds for Windows and macOS, the Linux binary has to be compiled from source. In this post, I’ll provide the steps I followed to compile a MultiMarkdown binary for Fedora 27.

Read more...

Using Docker Machine with KVM and Libvirt

Docker Machine is, in my opinion, a useful and underrated tool. I’ve written before about using Docker Machine with various services/providers; for example, see this article on using Docker Machine with AWS, or this article on using Docker Machine with OpenStack. Docker Machine also supports local hypervisors, such as VMware Fusion or VirtualBox. In this post, I’ll show you how to use Docker Machine with KVM and Libvirt on a Linux host (I’m using Fedora 27 as an example).

Read more...

Happy Thanksgiving 2017

In the US, today (Thursday, November 23) is Thanksgiving. I’d like to take a moment to reflect on the meaning of Thanksgiving.

Read more...

Installing Older Docker Client Binaries on Fedora

Sometimes there’s a need to have different versions of the Docker client binary available. On Linux this can be a bit challenging because you don’t want to install a “full” Docker package (which would also include the Docker daemon); you only need the binary. In this article, I’ll outline a process I followed to get multiple (older) versions of the Docker client binary on my Fedora 27 laptop.

Read more...

Installing Postman on Fedora 27

I recently had a need to install the Postman native app on Fedora 27. The Postman site itself only provides a link to the download and a rather generic set of instructions for installing the Postman native app (a link to these instructions for Ubuntu 16.04 is also provided). There were not, however, any directions for Fedora. Hence, I’m posting the steps I took to set up the Postman native app on my Fedora 27 laptop.

Read more...

Making AWS re:Invent More Family-Friendly

AWS re:Invent is just around the corner, and Spousetivities will be there to help bring a new level of family friendliness to the event. If you’re thinking of bringing a spouse, partner, or significant other with you to Las Vegas, I’d encourage you to strongly consider getting him or her involved in Spousetivities.

Read more...

Older Posts

Find more posts by browsing the post categories, content tags, or site archives pages. Thanks for visiting!