Scott's Weblog The weblog of an IT pro focusing on cloud computing, Kubernetes, Linux, containers, and networking

Technology Short Take 166

Welcome to Technology Short Take #166! I’ve been collecting links for the last few weeks, and now it’s time to share them with all of you. There are some familiar names in the links below, but also some newcomers—and I’m really excited to see that! I’m constantly on the lookout for new sources (if you have a site you think I should check out, hit me up—my contact info is at the bottom of this post!). But enough of that, let’s get on with the content. Enjoy!

Networking

Servers/Hardware

Security

  • Jeff Warren discusses a potential way for malicious players to bypass multi-factor authentication, aka the “Pass the Cookie” attack.
  • Aditya Patel takes a closer look at AWS’ recent announcement to enable server-side encryption (SSE) on S3 by default, and whether this new default setting offers any real improvement in security posture. I won’t spoil you by sharing his conclusion; go read the article (which is really well-written, in my opinion) to find out for yourself.
  • Alberto Pellitteri with Sysdig discusses SCARLETEEL, an operation conducted by an attacker that leveraged many of the tools found in modern cloud environments: Kubernetes, Terraform, and AWS. I highly recommend reviewing the article and considering what takeaways apply to your environment, if any.
  • Martin Smolár of ESET provides the first public analysis of a UEFI bootkit that is capable of bypassing UEFI Secure Boot, a bootkit known as BlackLotus.

Cloud Computing/Cloud Management

  • This article on using Open Policy Agent (OPA) as a custom Lambda authorizer for the AWS API Gateway was informative and helpful. It did underscore something for me, though: I need to improve my coding skills.
  • Do you need Argo CD? This article by Kirill Shirinkin provides, in the author’s words, “some guidelines that will help you to assess if Argo CD makes sense for your setup”.

Operating Systems/Applications

Storage

Virtualization

I don’t have any career/soft skills links for you this time, so that’s all for now! I hope that I’ve included something that you’ll find useful. As always, I invite your feedback on this post or any post on my site; feel free to reach out to me on Twitter or find me on Mastodon. I’m also present in a number of Slack communities, and you’re welcome to contact me directly there as well. Thank you for reading!

Creating a Talos Linux Cluster on AWS with Pulumi

Talos Linux is a Linux distribution purpose-built for running Kubernetes. The Talos web site describes Talos Linux as “secure, immutable, and minimal.” All system management is done via an API; there is no SSH access, no shell, and no console. In this post, I’ll share how to use Pulumi to automate the creation of a Talos Linux cluster on AWS.

I chose to write my Pulumi program in Go, but you could—of course—choose to write it in any language that Pulumi supports (JavaScript/TypeScript, Python, one of the .NET languages, Java, or even YAML). I’ve made the Pulumi program available via this GitHub repository. It’s based on these instructions for standing up Talos Linux on AWS.

The Pulumi program has four major sections:

  1. First, it creates the underlying base infrastructure needed for a Talos Linux cluster to run. This includes a VPC (and all the assorted other pieces, like subnets, gateways, routes, and route tables) and a load balancer. The load balancer is needed for the Kubernetes control plane, which we will bootstrap later in the program. This portion also creates the EC2 instances for the control plane.
  2. Next, it uses the Talos Pulumi provider to generate the Talos configuration that will be applied to the instances after they are created. This configuration needs information from step 1; specifically, it needs the DNS name of the load balancer and the IP addresses of the control plane nodes.
  3. Third, it applies the correct configuration to the control plane nodes, and launches some EC2 instances to serve as worker nodes in the Kubernetes cluster. The program then takes the Talos configuration from step 2 and applies it to the worker nodes.
  4. The fourth and final step is to bootstrap the cluster.

Let’s examine each of these sections in a bit more detail.

Creating the AWS Infrastructure

The first part of the Pulumi program (running through about line 197) creates the necessary AWS infrastructure. The program leverages Crosswalk for AWS to simplify the creation of the underlying VPC and associated components. Most of the code involves creating security groups and security group rules.

This part of the program also creates the load balancer that will be used for the Kubernetes API.

Finally, the program launches three EC2 instances using one of the official Talos AMIs (you have to supply the AMI ID as a configuration value). The IDs and the private IP addresses for these EC2 instances are stored in two different arrays; we’ll need this information later.

Note that you could extend the Pulumi Go program to look up the correct AMI for you so that this information doesn’t need to be passed in as a configuration value via pulumi config. I’ll leave this as an exercise for the reader.

Creating the Talos Configuration

The next part of the Pulumi program builds the Talos configuration, using the prerelease Pulumi provider and information (outputs) from the resources created earlier. Specifically, the Talos configuration uses the DNS name of the load balancer created earlier, and the IP addresses of the EC2 instances (which will become the control plane nodes for the Kubernetes cluster).

There are three parts to the configuration:

  1. A client configuration file, which the talosctl command-line utility uses to connect to the Talos Linux nodes
  2. Machine configuration for the control plane nodes
  3. Machine configuration for the worker nodes

To install the prerelease provider, you can follow these instructions.

Configuring the Nodes

Once the Talos provider builds the correct machine configurations, the program then applies the configuration to each of the control plane nodes (worker nodes are handled later). It does this by looping through the array with the instance IP addresses and applying the configuration to each instance.

Once the machine configuration has been applied to the control plane nodes, three more EC2 instances are launched for worker nodes, and then the correct machine configuration is applied to those nodes (using the mechanism of looping over an array).

Bootstrapping the Cluster

Finally, it comes down to bootstrapping the cluster, which—again using the Talos provider for Pulumi—is accomplished with calling talos.NewTalosMachineBootstrap against the first control plane node to be created. The Pulumi program will complete after this; it doesn’t wait for the bootstrap process to complete (which will take a few more minutes).

At this point, you can use pulumi stack output to retrieve a configuration file for the talosctl command-line utility, and then run talosctl health to watch the cluster bootstrap. Once the cluster is up and running, talosctl kubeconfig will get you a Kubeconfig file you can use to access the Kubernetes API via kubectl. Nice!

Additional Resources

All of the Pulumi code is available on GitHub in this repository. Please note that I tested the code and it works for me, but it is supplied “as-is”. Don’t use this for your production environment without performing your own testing and validation!

I hope you’ve found this post helpful. If you have questions, you’re welcome to open an issue in the GitHub repository, or you can contact me directly. It’s not hard to get in touch with me—you can find me on Twitter, on Mastodon, and on various Slack communities (including the Kubernetes and Pulumi Slack communities).

Technology Short Take 165

Welcome to Technology Short Take #165! Over the last few weeks, I’ve been collecting articles I wanted to share with readers on major areas in technology: networking, security, storage, virtualization, cloud computing, and OSes/applications. This particular Technology Short Take is a tad heavy on cloud computing, but there’s a decent mix of other articles as well. Enjoy!

Networking

  • For a deeper understanding of Kubernetes networking, and in particular the role played by kube-proxy, I highly recommend this post by Arthur Chiao. There is a ton of information here!
  • Denis Mulyalin shows how to use Nornir, Salt, and NetBox to template your network tests. Now if Denis’ site just had a discoverable RSS feed…

Security

  • Aeva Black and Gil Yehuda tackle the conundrum of open source security.
  • If you haven’t looked at Teri Radichel’s series of posts on automating cybersecurity metrics (ACM), you should. There’s quite a bit of good information there.
  • This post on Cedar—a new policy language developed by AWS—is an interesting read. I’m curious as to the constraints that led AWS to develop a new policy language versus using something like Rego (part of Open Policy Agent); this isn’t something the article touches upon.

Cloud Computing/Cloud Management

Operating Systems/Applications

  • I don’t do a whole lot in application development (yet), but the idea of declarative database schemas is a pretty cool idea. Too bad they implemented a proprietary DSL (domain-specific language) instead of allowing users to use a general-purpose programming language.
  • It’s a fairly straightforward process, but here are some instructions for installing Docker and Docker Compose on Amazon Linux 2.
  • I didn’t really understand Dagger until I read this. I still don’t necessarily fully understand Dagger, but at least I have some idea about how it can be used. (I’m sure it’s no surprise that I am a fan of using full-featured programming languages to do things.)
  • As fate (luck?) would have it, I found this article by Engin Diri on using kube2pulumi right around the same time I actually needed to use it to convert some Kubernetes YAML into a Pulumi TypeScript program.
  • This post outlines some recent gVisor improvements in file system performance.

Career/Soft Skills

  • This is a slightly older post about taking the CKA, but what I really liked was Brandon’s use of the graphic at the top of the post that talks about failure being part of success. It’s so true—we learn and grow through failure (or should, at least). As you go out and start tackling new things, don’t let failure stop you; use the failure to prepare you for the next attempt.
  • I love this three-part (so far) series by Adrian Hornsby on skills that make a big difference (part 1, part 2, and part 3). Part 3 is a strong reminder of what I mentioned in the first bullet in this section: don’t let failure stop you, use it to prepare for the next attempt. As Adrian quoted from Edison: “Every time I fail, I am getting closer to the truth.”
  • I didn’t know it was called this, but this technique is something I’ve used for a while now. Switching between active projects when your focus is flagging, especially when one of them is a learning project, can be useful, but like all things it needs to be managed carefully.

I have a ton more links I could share, but in the interest of keeping the length of this post manageable I’ll wrap it up here. I hope that I managed to include something useful for you here; if not, please let me know! Are there technology areas you’d like to see more coverage of? Or are there areas you’d like to see less coverage? Those of you that have feedback on questions like this are invited to contact me! You can reach me on Twitter, on Mastodon, via e-mail (my address isn’t terribly hard to find), or through Slack (I’m active in a number of different Slack communities). Thanks for reading!

Stage Manager is Incomplete

I’ve been using macOS Stage Manager off and on for a little while now. In Stage Manager, I can see the beginnings of what might be a very useful paradigm for desktop computing. Unfortunately, in its current incarnation, I believe Stage Manager is incomplete.

Note that I haven’t yet tried Stage Manager on my iPad; my comments here apply only to the macOS implementation.

For those of you who haven’t yet tried Stage Manager yet, here’s a screenshot of my desktop, taken while I was writing this blog post:

Desktop screenshot of macOS with Stage Manager enabled

I’ll draw your attention to the list of “recently used applications” on the left side of the screen. That’s the “Cast” (a term used by Howard Oakley in his great introductory article on Stage Manager). As you can see in this screenshot, the Cast supports application groups—like having Slack and Mail grouped together—as well as single applications. This allows you to easily switch between groups of applications simply by clicking on the preview in the Cast (which, using Howard’s terminology, moves the application or applications to the Stage).

This is the glimmer of a useful paradigm that I see in Stage Manager: being able to assemble groups of applications that you often use together and being able to easily switch between those groups of applications.

Unfortunately, the Cast of recent applications shown on the left of the screen is as far as Apple went with that paradigm, and this is why I say that Stage Manager is incomplete. When I use Stage Manager, I’m left asking questions like:

  • Why not extend Cmd-Tab to switch between application groups? The Cmd-Tab application switcher isn’t what I would call “Stage Manager” aware, so it just displays individual applications (not the Stage Manager application groups you’ve created). Yes, it still switches the applications between Cast and Stage, but why not display the application groups in the switcher?
  • Why not extend the Dock to show application groups? Again, I feel like Apple needs to move the focus away from individual applications to application groups (which may be a single application). Why not extend the Dock to show application groups? In fact, why not make the Dock the mechanism to show the Cast, instead of the previews on the left? Or, if you insist on keeping the Cast as thumbnails on the left, why not get rid of the Dock entirely? Adopt something from GNOME and put Launchpad behind a hotkey for launching apps.
  • Why not merge Spaces and Stage Manager? Both of these features seem aimed at helping users manage lots of windows—why not merge them? Give us the ability to assign an application to an “App Group”, where it will be grouped together with other applications in the Cast when it is launched (much in the same way you can assign an application to a Space).

I do believe that Stage Manager has the potential to be enormously useful. In its current form, though, it just feels incomplete. If you do decide to stick with it, though, Howard Oakley’s article on Stage Manager workflows would be useful, I think.

Thanks for reading! Have opinions about what I’ve written? Contact me on Twitter or on Mastodon and tell me what you think!

Installing the Prerelease Pulumi Provider for Talos

Normally, installing a Pulumi provider is pretty easy; you run pulumi up and the provider gets installed automatically. Worst case scenario, you can install the provider using pulumi plugin install. However, sometimes things have to be done manually. In this post, I’ll walk you through installing a prerelease provider—in this case, the prerelease Pulumi provider for Talos Linux—using both pulumi plugin install as well as using a manual process.

Currently, the prerelease provider for Talos Linux (found in this GitHub repository) can’t be installed automatically when running pulumi up. So, to use the Talos Linux provider, you’ll need to install it before running pulumi up.

To install it using pulumi plugin install, you need to know the version you want to install. As of this writing, the latest release was v0.1.0-beta.0. Armed with that information, you can install the plugin with this command:

pulumi plugin install resource talos v0.1.0-beta.0 --server github://api.github.com/siderolabs/pulumi-provider-talos

If the GitHub repository had been named “pulumi-talos” instead of “pulumi-provider-talos”, the previous command could be shortened:

pulumi plugin install resource talos v0.1.0-beta.0 --server github://api.github.com/siderolabs

This is because the pulumi plugin install command will automatically look for a repository named pulumi-<provider>.

If, for whatever reason, you wish to install the plugin manually, the process looks like this:

  1. Download the latest release of the Talos provider from the GitHub Releases page. This will download a tarred and gzipped archive.
  2. The plugin files need to go into a specific subdirectory under ~/.pulumi/plugins. Navigate to that directory, and create a subdirectory whose name corresponds to the version of the Talos provider. For example, if the version downloaded is v0.1.0-beta.0, then the name of the new subdirectory should be resource-talos-v0.1.0-beta.0.
  3. Extract the contents of the release downloaded in step 1 to the new subdirectory created in step 2. You can use tar -C ~/.pulumi/plugins/<new-subdirectory> -xzf <release-archive> to extract directly into the new subdirectory created in step 2.
  4. Run pulumi plugin ls. You should now see the Talos provider listed.

Once you see the provider listed with pulumi plugin ls, you’re ready to run pulumi up with programs that leverage the Talos provider.

If you have any questions about installing the plugin, please feel free to reach out to me; I’ll do my best to help. You can contact me on Twitter, on Mastodon, or via one of the various Slack communities I frequent. Enjoy using your new Talos provider with Pulumi!

UPDATE 2023-02-10: I’ve updated the post with some additional information provided via Ringo De Smet, a colleague of mine at Pulumi. Thank you, Ringo!

Recent Posts

Automating Docker Contexts with Pulumi

Since I switched my primary workstation to an M1-based MacBook Pro (see my review here), I’ve starting using temporary AWS EC2 instances for compiling code, building Docker images, etc., instead of using laptop-local VMs. I had an older Mac Pro (running Fedora) here in my home office that formerly filled that role, but I’ve since given that to my son (he’s a young developer who wanted a development workstation). Besides, using EC2 instances has the benefit of access when I’m away from my home office. I use Pulumi to manage these instances, and I extended my Pulumi code to also include managing local Docker contexts for me as well. In this post, I’ll share the solution I’m using.

Read more...

Technology Short Take 164

Welcome to Technology Short Take #164! I’ve got another collection of links to articles on networking, security, cloud, programming, and career development—hopefully you find something useful!

Read more...

Technology Short Take 163

Welcome to Technology Short Take #163, the first of 2023! If you’re new to this site, the Technology Short Takes are essentially “link lists”—I collect links and articles about various technologies and I share them about every 3-4 weeks (sometimes more frequently). I’ll often add a bit of commentary here and there, but the real focus is the information in the linked articles. But enough of this, let’s get on with it! Here’s hoping you find something useful here.

Read more...

A Depth Year in 2023

Off and on for a number of years, I published a “projects for the coming year” post and a “report card for last year’s projects” post (you can find links to all of these here). Typically, the project list was composed of new things I would learn and/or new things I would create or do. While there’s nothing wrong with this sort of thing—not at all!—I came across an idea while reading that I’ve decided I’ll adopt for 2023: a depth year.

Read more...

Technology Short Take 162

Welcome to Technology Short Take #162! It’s taken me a bit longer than I would have liked to get this post assembled, but it’s finally here. Hopefully I’ve managed to find something you’ll find useful! As usual, the links below are organized by technology area/discipline, and I’ve added a little bit of commentary to some of the links where it felt necessary. Enjoy!

Read more...

Technology Short Take 161

Welcome to Technology Short Take #161! It’s been a little over a month since the last Technology Short Take, although the Full Stack Journey recently did an “Audio Edition” of a Technology Short Take that you should probably check out. In any case, I’ve spent the last month collecting links to articles and tutorials from around the web on all the various technologies that us IT folk are likely to encounter in our day-to-day adventures. I hope there’s something here that you find useful!

Read more...

Streamlining the User Experience for Accessing AKS Clusters

Lately I’ve been spending a little bit of time building Pulumi programs to assist with standing up Azure Kubernetes Service (AKS) clusters. I’ve learned a pretty fair amount about Azure and AKS along the way, as expected, but I was taken aback by the poor user experience (in my opinion) when it came to accessing the AKS clusters once they’d been established. In this post, I’ll share a small tweak you can make that will, in most cases, make accessing your AKS clusters a great deal smoother.

Read more...

Technology Short Take 160

Welcome to Technology Short Take #160! This time around, my list of links and articles is a tad skewed toward cloud computing/cloud management, but I’ve still managed to pull together some links on other topics that readers will hopefully find useful. For example, did you know about the secret macOS network quality tool? You didn’t? Lucky for you there’s a link to an article about it below. Read on to get all the details!

Read more...

Referencing Configuration Values in Pulumi YAML

Lately I’ve been doing a fair amount of work with Pulumi’s YAML support (see this blog post announcing it), and I recently ran into a situation where I wanted to read in and use a configuration value (set via pulumi config). When using one of Pulumi’s supported programming languages, like TypeScript or Python or Go, this is pretty easy. It’s also easy in YAML, but not as intuitive as I originally expected. In this post, I’ll share how to read in and use a configuration value when using Pulumi YAML.

Read more...

Managing AWS Key Pairs with Pulumi and Go

As I was winding down things at Kong and getting ready to transition to Pulumi (more information on why I moved to Pulumi here), I casually made the comment on Twitter that I needed to start managing my AWS key pairs using Pulumi. When the opportunity arose last week, I started doing exactly that! In this post, I’ll show you a quick example of how to use Pulumi and Go to declaratively manage AWS key pairs.

Read more...

Technology Short Take 159

Welcome to Technology Short Take #159! If you’re interested in finding some links to articles around the web on topics like WASM, Git, Sigstore, or EKS—among other things—then you’ve come to the right place. I’ve spent the last few weeks collecting articles I think you’ll find useful, gleaning them from the depths of Twitter, RSS feeds, Reddit, and Slack. Enjoy, and never stop learning!

Read more...

Jumping Off Cliffs

For quite a few years, I’ve had this desktop wallpaper that I really love. I don’t even remember where I got it or where it came from, so I can’t properly attribute it to anyone. I use this wallpaper from time to time when I want to be reminded to challenge myself, to learn new things, and to step outside of what is comfortable in order to explore the as-yet-unknown. Looking at this wallpaper on my desktop a little while ago, I realized that I may have started taking the inspirational phrase on this wallpaper for granted, instead of truly applying it to my life.

Read more...

Technology Short Take 158

Welcome to Technology Short Take #158! What do I have in store for you this time around? Well, you’ll have to read the whole article to find out for sure, but I have links to articles on…well, lots of different topics! DNS, BGP, hardware-based security, Kubernetes, Linux—they’re all in here. Hopefully I’ve managed to find something useful for someone.

Read more...

Revisiting X.509 Certificates in Kubeconfig Files

In 2018, I wrote an article on examining X.509 certificates embedded in Kubeconfig files. In that article, I showed one way of extracting client certificate data from a Kubeconfig file and looking at the properties of the client certificate data. While there’s nothing technically wrong with that article, since then I’ve found another tool that makes the process a tad easier. In this post, I’ll revisit the topic of examining embedded X.509v3 certificates in Kubeconfig files.

Read more...

Posts from the Past, August 2022

I thought I might start highlighting some older posts here on the site through a semi-regular “Posts from the Past” series. I’ll start with posts published in the month of August through the years. Here’s hoping you find something that is useful (or perhaps entertaining, at least)!

Read more...

Older Posts

Find more posts by browsing the post categories, content tags, or site archives pages. Thanks for visiting!