azure-labs: lift and shift Azure VMs between resource groups

README.md

@@ -115,6 +115,16 @@ When you are done, `cd ..` out of the folder and run `labagent rag destroy --dir
 
 On Windows PowerShell use `$HOME\rag-lab` and `cd $HOME\rag-lab`. If you installed `rag-local` instead, add `pip install jupyterlab`, or open the notebooks in VS Code and pick the Python environment you installed into as the kernel. See [docs/rag-local.md](docs/rag-local.md).
 
+## Azure Labs: move Azure VMs to another resource group
+
+Lift and shift VMs, disks and network from one Azure resource group to another with snapshots, or snapshot a dead VM or disk into a group of its own. It never changes the source.
+An alternative for people leaving Azure Lab Services (retiring June 28, 2027), not Azure Lab Services itself. [Guide](docs/azure-labs.md)
+
+```bash
+labagent launch azure-labs --name move1 --param source_rg=azure-labs-legacy --param target_rg=azure-labs-new
+labagent destroy azure-labs --name move1        # deletes only what that run created
+```
+
 ## AWS sandboxes
 
 ```bash

docs/azure-labs.md

@@ -0,0 +1,99 @@
+# Azure Labs: move Azure VMs from one resource group to another
+
+A simple lift and shift for Azure VMs. It copies the VMs, their disks and their network into another resource group, and it never changes the source.
+It can also just snapshot disks, for an old VM or disk you want to keep somewhere else.
+
+This is an alternative for people moving off Azure Lab Services (retiring June 28, 2027). It is not Azure Lab Services and not from Microsoft.
+Lab VMs are ordinary Azure VMs, so this works for any VM in a resource group, not only lab VMs.
+
+## What you need
+
+- The Azure CLI, signed in: `az login`.
+- Your subscription ID in the allow list. labagent refuses to run on any other subscription.
+
+```bash
+export LABAGENT_ALLOWED_AZURE_SUBSCRIPTIONS=$(az account show --query id -o tsv)
+```
+
+PowerShell: `$env:LABAGENT_ALLOWED_AZURE_SUBSCRIPTIONS = az account show --query id -o tsv`
+
+## Copy VMs to another resource group
+
+```bash
+labagent launch azure-labs --name move1 --param source_rg=azure-labs-legacy --param target_rg=azure-labs-new
+```
+
+| Parameter | Meaning |
+|---|---|
+| `source_rg` | Group to copy from. It is only read. |
+| `target_rg` | Group to copy into. Created if it does not exist. |
+| `vms` | Comma list of VM names. Default: every VM in the source group. |
+| `snapshot_only` | `true` copies only disk snapshots (no VMs, no network). With no `vms`, it also takes disks that no VM is using. |
+| `subscription` | Subscription ID or name. Default: the one `az login` selected. |
+
+What it does, in order: reads the source, checks vCPU quota, creates the target group if needed, snapshots every disk into the target,
+copies the virtual network, subnets, network security groups (with their rules), public IPs and NICs, creates disks from the snapshots,
+creates the VMs on those disks, and checks that each VM is provisioned. Every copied resource keeps its name and gets these tags:
+`labagent=1`, `pack=azure-labs`, `name`, `migrated_from`, `created_at`.
+
+Run it again after a failure: it skips what already exists and carries on. Add `--dry-run` to see the steps without running them.
+
+The new VMs boot the same operating system with the same accounts and data (a specialized copy), so no password is needed or created.
+They start running. The source VMs keep running too, and both count against your vCPU quota. Stop the sources, or copy fewer VMs at a time with `vms=`.
+
+## Snapshot an old VM or disk
+
+```bash
+labagent launch azure-labs --name old1 --param source_rg=azure-labs-legacy --param target_rg=azure-labs-archive --param snapshot_only=true
+```
+
+Snapshots land in the target group as `snap-<disk name>`. Create a disk or VM from one whenever you need it.
+
+## Delete
+
+```bash
+labagent destroy azure-labs --name move1
+```
+
+Deletes only what that migration created in the target group (found by its tags), dependants first. It deletes the target group too if labagent created it
+and nothing else is in it. Anything in the group without the tags is reported and left alone. The source group is never touched.
+
+## What it does not copy
+
+It stops before creating anything, and lists the reasons, if a VM is in an availability set or scale set, uses an unmanaged VHD,
+uses encrypted disks with your own keys, has a NIC with more than one IP configuration, or uses application security groups.
+
+- Public IP addresses change: you get a new address with the same name.
+- Data disk order is kept, but LUN numbers and caching settings are not.
+- VM extensions, managed identities, boot diagnostics settings and backups are not copied.
+- Same subscription and same region only. Copying across subscriptions is not built.
+- Managed images are not copied. Create a VM from the image, then snapshot that.
+
+## Try it on a test group
+
+Create a small legacy group first. Three B1s VMs fit a 10-vCPU quota twice over.
+
+```bash
+az group create -n azure-labs-legacy -l canadacentral
+az network nsg create -g azure-labs-legacy -n legacy-nsg
+az network nsg rule create -g azure-labs-legacy --nsg-name legacy-nsg -n allow-ssh --priority 100 --destination-port-ranges 22 --access Allow --protocol Tcp
+az network vnet create -g azure-labs-legacy -n legacy-vnet --address-prefixes 10.30.0.0/16 --subnet-name app --subnet-prefixes 10.30.1.0/24 --network-security-group legacy-nsg
+for i in 1 2 3; do
+  az vm create -g azure-labs-legacy -n legacy-vm$i --image Ubuntu2204 --size Standard_B1s --vnet-name legacy-vnet --subnet app \
+    --public-ip-address "" --nsg "" --admin-username azureuser --generate-ssh-keys
+done
+az vm disk attach -g azure-labs-legacy --vm-name legacy-vm1 --name legacy-vm1-data --new --size-gb 8
+az disk create -g azure-labs-legacy -n old-dead-disk --size-gb 4
+```
+
+Then run the copy, check the result, and delete it:
+
+```bash
+labagent launch azure-labs --name test1 --param source_rg=azure-labs-legacy --param target_rg=azure-labs-new
+az vm list -g azure-labs-new -d -o table            # 3 VMs, running
+az resource list -g azure-labs-legacy -o table      # unchanged
+labagent destroy azure-labs --name test1
+az group exists -n azure-labs-new                   # false
+```
+
+Clean up the test source with `az group delete -n azure-labs-legacy --yes`.

labagent/packs/azure_labs/__init__.py

@@ -0,0 +1 @@
+"""azure-labs: lift and shift Azure VMs from one resource group to another with snapshots. Not Azure Lab Services."""

labagent/packs/azure_labs/azcli.py

@@ -0,0 +1,148 @@
+"""Thin wrapper over the `az` CLI. The runner is injectable so tests never touch Azure. Error text is redacted."""
+from __future__ import annotations
+
+import json
+import re
+import shutil
+import subprocess
+from typing import Any, Dict, List, Optional
+
+from ..base import PackError
+
+SECRET = re.compile(r"(eyJ[\w-]{20,}\.[\w-]{10,}\.[\w-]{10,}|(?i:password|secret|key|token)\S*\s*[=:]\s*\S+)")
+
+
+def redact(text: str, tail: int = 6) -> str:
+    return SECRET.sub("<redacted>", "\n".join(str(text).strip().splitlines()[-tail:]))[:600]
+
+
+def tag_args(tags: Dict[str, str]) -> List[str]:
+    return ["--tags", *[f"{k}={v}" for k, v in tags.items()]]
+
+
+class AzCli:
+    def __init__(self, subscription: Optional[str] = None, run=subprocess.run, binary: Optional[str] = None):
+        self.subscription = subscription
+        self._run = run
+        self._bin = binary
+
+    def binary(self) -> str:
+        b = self._bin or shutil.which("az")
+        if not b:
+            raise PackError("Azure CLI not found. Install it (https://learn.microsoft.com/cli/azure/install-azure-cli) and run: az login")
+        return b
+
+    def json(self, *args: str) -> Any:
+        cmd = [self.binary(), *args, "--only-show-errors", "-o", "json"]
+        if self.subscription:
+            cmd += ["--subscription", self.subscription]
+        p = self._run(cmd, capture_output=True, text=True, encoding="utf-8", errors="replace")
+        if p.returncode != 0:
+            raise PackError(f"az {' '.join(args[:2])} failed: {redact(p.stderr or p.stdout)}")
+        return json.loads(p.stdout) if (p.stdout or "").strip() else {}
+
+    # ---- account and capacity ------------------------------------------------------------------------------------------
+    def account(self) -> Dict[str, Any]:
+        try:
+            return self.json("account", "show")
+        except PackError as e:
+            raise PackError(f"could not read the Azure login. Run: az login ({e})") from None
+
+    def usage(self, region: str) -> Dict[str, Dict[str, int]]:
+        rows = self.json("vm", "list-usage", "-l", region)
+        return {r["name"]["value"]: {"used": int(r["currentValue"]), "limit": int(r["limit"])} for r in rows}
+
+    def skus(self, region: str) -> Dict[str, Dict[str, Any]]:
+        """Every VM size in the region: family, vCPUs, restricted. One call per run (az filters on the client, so it takes about a minute)."""
+        cache = self.__dict__.setdefault("_sku_cache", {})
+        if region not in cache:
+            rows = self.json("vm", "list-skus", "-l", region, "--resource-type", "virtualMachines", "--query",
+                             "[].{name:name,family:family,restricted:length(restrictions),vcpus:capabilities[?name=='vCPUs'].value|[0]}")
+            cache[region] = {r["name"]: r for r in rows}
+        return cache[region]
+
+    # ---- groups and resources -----------------------------------------------------------------------------------------
+    def group_exists(self, name: str) -> bool:
+        return bool(self.json("group", "exists", "-n", name))
+
+    def group_show(self, name: str) -> Optional[Dict[str, Any]]:
+        return self.json("group", "show", "-n", name) if self.group_exists(name) else None
+
+    def group_create(self, name: str, region: str, tags: Dict[str, str]) -> None:
+        self.json("group", "create", "-n", name, "-l", region, *tag_args(tags))
+
+    def group_delete(self, name: str) -> None:
+        self.json("group", "delete", "-n", name, "--yes")
+
+    def resources(self, group: str) -> List[Dict[str, Any]]:
+        return self.json("resource", "list", "-g", group) or []
+
+    def resource_delete(self, rid: str) -> None:
+        self.json("resource", "delete", "--ids", rid)
+
+    def show(self, rid: str) -> Dict[str, Any]:
+        return self.json("resource", "show", "--ids", rid)
+
+    # ---- reading the source (read-only) ---------------------------------------------------------------------------------
+    def list_vms(self, group: str) -> List[Dict[str, Any]]:
+        return self.json("vm", "list", "-g", group) or []
+
+    def list_disks(self, group: str) -> List[Dict[str, Any]]:
+        return self.json("disk", "list", "-g", group) or []
+
+    def vm_state(self, group: str, name: str) -> Dict[str, Any]:
+        return self.json("vm", "show", "-d", "-g", group, "-n", name)
+
+    # ---- creating in the target ---------------------------------------------------------------------------------------
+    def snapshot_create(self, group: str, name: str, source_id: str, location: str, tags: Dict[str, str], hyper_v: Optional[str] = None) -> None:
+        extra = ["--hyper-v-generation", hyper_v] if hyper_v else []
+        self.json("snapshot", "create", "-g", group, "-n", name, "--source", source_id, "-l", location, "--incremental", "false", *extra, *tag_args(tags))
+
+    def disk_create(self, group: str, name: str, snapshot_id: str, sku: str, location: str, tags: Dict[str, str],
+                    zone: Optional[str] = None, hyper_v: Optional[str] = None, os_type: Optional[str] = None) -> None:
+        extra = (["--zone", zone] if zone else []) + (["--hyper-v-generation", hyper_v] if hyper_v else []) + (["--os-type", os_type] if os_type else [])
+        self.json("disk", "create", "-g", group, "-n", name, "--source", snapshot_id, "--sku", sku, "-l", location, *extra, *tag_args(tags))
+
+    def nsg_create(self, group: str, name: str, location: str, tags: Dict[str, str]) -> None:
+        self.json("network", "nsg", "create", "-g", group, "-n", name, "-l", location, *tag_args(tags))
+
+    def nsg_rule_create(self, group: str, nsg: str, rule: Dict[str, Any]) -> None:
+        args = ["network", "nsg", "rule", "create", "-g", group, "--nsg-name", nsg, "-n", rule["name"], "--priority", str(rule["priority"]),
+                "--direction", rule["direction"], "--access", rule["access"], "--protocol", rule["protocol"]]
+        for flag, key in (("--source-address-prefixes", "source_addresses"), ("--source-port-ranges", "source_ports"),
+                          ("--destination-address-prefixes", "destination_addresses"), ("--destination-port-ranges", "destination_ports")):
+            args += [flag, *rule[key]]
+        self.json(*args)
+
+    def vnet_create(self, group: str, name: str, prefixes: List[str], subnet: str, subnet_prefix: str, location: str, tags: Dict[str, str]) -> None:
+        self.json("network", "vnet", "create", "-g", group, "-n", name, "--address-prefixes", *prefixes, "--subnet-name", subnet,
+                  "--subnet-prefixes", subnet_prefix, "-l", location, *tag_args(tags))
+
+    def subnet_create(self, group: str, vnet: str, name: str, prefix: str) -> None:
+        self.json("network", "vnet", "subnet", "create", "-g", group, "--vnet-name", vnet, "-n", name, "--address-prefixes", prefix)
+
+    def subnet_set_nsg(self, group: str, vnet: str, subnet: str, nsg: str) -> None:
+        self.json("network", "vnet", "subnet", "update", "-g", group, "--vnet-name", vnet, "-n", subnet, "--network-security-group", nsg)
+
+    def public_ip_create(self, group: str, name: str, location: str, tags: Dict[str, str]) -> None:
+        self.json("network", "public-ip", "create", "-g", group, "-n", name, "-l", location, "--sku", "Standard", "--allocation-method", "Static", *tag_args(tags))
+
+    def nic_create(self, group: str, name: str, vnet: str, subnet: str, location: str, tags: Dict[str, str],
+                   nsg: Optional[str] = None, public_ip: Optional[str] = None, accelerated: bool = False) -> None:
+        args = ["network", "nic", "create", "-g", group, "-n", name, "--vnet-name", vnet, "--subnet", subnet, "-l", location]
+        if nsg:
+            args += ["--network-security-group", nsg]
+        if public_ip:
+            args += ["--public-ip-address", public_ip]
+        if accelerated:
+            args += ["--accelerated-networking", "true"]
+        self.json(*args, *tag_args(tags))
+
+    def vm_create(self, group: str, name: str, os_disk: str, os_type: str, size: str, nics: List[str], data_disks: List[str],
+                  location: str, tags: Dict[str, str], zone: Optional[str] = None) -> None:
+        args = ["vm", "create", "-g", group, "-n", name, "--attach-os-disk", os_disk, "--os-type", os_type, "--size", size, "--nics", *nics, "-l", location]
+        if data_disks:
+            args += ["--attach-data-disks", *data_disks]
+        if zone:
+            args += ["--zone", zone]
+        self.json(*args, *tag_args(tags))

labagent/packs/azure_labs/lifecycle.py

@@ -0,0 +1,68 @@
+"""vCPU quota arithmetic and tag-only teardown."""
+from __future__ import annotations
+
+from typing import Any, Callable, Dict
+
+from ..base import PackError
+from . import safety
+
+# Delete in this order so dependants go first.
+ORDER = ["microsoft.compute/virtualmachines", "microsoft.network/networkinterfaces", "microsoft.compute/disks", "microsoft.compute/snapshots",
+         "microsoft.network/publicipaddresses", "microsoft.network/virtualnetworks", "microsoft.network/networksecuritygroups"]
+ROUNDS = 4
+
+
+def check_quota(usage: Dict[str, Dict[str, int]], needs: Dict[str, int], region: str) -> None:
+    """`needs` is vCPUs per VM family. Also checks the regional total."""
+    def free(key: str) -> int:
+        u = usage.get(key)
+        return (u["limit"] - u["used"]) if u else 0
+    total = sum(needs.values())
+    short = [f"total regional vCPUs: need {total}, free {free('cores')}"] if free("cores") < total else []
+    short += [f"{fam}: need {n}, free {free(fam)}" for fam, n in sorted(needs.items()) if free(fam) < n]
+    if short:
+        raise PackError(f"not enough vCPU quota in {region} ({'; '.join(short)}). The source VMs still count while they run: stop them, "
+                        "migrate fewer VMs at once (vms=a,b), or ask for more in the Azure portal under Subscriptions > Usage + quotas.")
+
+
+def _rank(rtype: str) -> int:
+    t = rtype.lower()
+    return ORDER.index(t) if t in ORDER else len(ORDER)
+
+
+def destroy_owned(az: Any, group: str, name: str, log: Callable[[str], None] = lambda m: None) -> Dict[str, Any]:
+    """Delete only the resources in `group` that carry this migration's tags, then the group if labagent created it and it is empty.
+
+    Raises PackError, deleting nothing more, when something without these tags is left. The source group is never passed here."""
+    info = az.group_show(group)
+    if info is None:
+        return {"group_deleted": False, "resources_deleted": 0}
+    deleted = 0
+    for _ in range(ROUNDS):
+        mine = [r for r in az.resources(group) if safety.owned(r.get("tags"), name) and r["type"].count("/") == 1]
+        if not mine:
+            break
+        failed = 0
+        for r in sorted(mine, key=lambda x: _rank(x["type"])):
+            try:
+                az.resource_delete(r["id"])
+                deleted += 1
+                log(f"  deleted {r['type'].split('/')[-1]} {r['name']}")
+            except PackError:
+                failed += 1                          # usually a dependency that is not gone yet: the next round retries
+        if not failed:
+            break
+    left = [r for r in az.resources(group) if r["type"].count("/") == 1]
+    mine_left = [r["name"] for r in left if safety.owned(r.get("tags"), name)]
+    foreign = [r["name"] for r in left if not safety.owned(r.get("tags"), name)]
+    if mine_left:
+        raise PackError(f"could not delete {', '.join(sorted(mine_left))} in {group}. Run destroy again.")
+    created = (info.get("tags") or {}).get("created_by_labagent") == "1"
+    if foreign:
+        if created:
+            raise PackError(f"{len(foreign)} resource(s) in {group} do not carry this migration's tags and were not deleted: "
+                            f"{', '.join(sorted(foreign))}. Remove them yourself, then run destroy again.")
+        return {"group_deleted": False, "resources_deleted": deleted}
+    if created:
+        az.group_delete(group)
+    return {"group_deleted": created, "resources_deleted": deleted}