Sårbarhet i Veeam Agent for Windows

Veeam Agent for Microsoft Windows Vulnerability (CVE-2024-29853) är en allvarlig sårbarhet som har upptäckts i Veeam Agent for Microsoft Windows. Låt mig ge dig en detaljerad sammanfattning:

Uppgradering av fristående vs managerad Veeam Agent-installation

När det gäller Veeam Agent for Microsoft Windows (VAW) finns det två huvudsakliga driftlägen: antingen fristående eller managerad. Om VAW hanteras av Veeam Backup & Replication (VBR) eller Veeam Service Provider Console (VSPC) måste VAW-installationen uppgraderas av den server som hanterar den. Att direkt uppgradera en VAW-installation som VBR hanterar kan leda till kompatibilitetsproblem om den installerade VAW-versionen är högre än vad VBR-programvaran förväntar sig.

Här är några riktlinjer för uppgradering:

  1. Fristående driftläge (utan koppling mot VBR eller Cloud Connect-repository):
    • I detta läge kan VAW uppgraderas direkt på den dator där den är installerad.
  2. Fristående driftläge (med koppling mot VBR eller Cloud Connect-repository):
    • Om VAW är konfigurerad för fristående driftläge men kopplat mot ett VBR- eller Cloud Connect-repositort måste mål-VBR eller Cloud Connect-installationen uppgraderas till en version som stöder den nya VAW-versionen innan VAW uppgraderas direkt.
  3. Managerat driftläge (VBR hanterar VAW):
    • Om en VBR-server hanterar VAW måste du först uppgradera VBR och sedan uppgradera de hanterade VAW-installationerna.
  4. Managerat driftläge (VSPC hanterar VAW):
    • Om en VSPC-server hanterar VAW måste VAW-installationen uppgraderas via VSPC-konsolen.

Replica seeding to vCloud Director

One of the many use cases for Veeam Backup & Replication is disaster recovery, as the name of the product suggests it can certainly replicate virtual machines from a production environment to a secondary- or disaster recovery environment. While it is a very straight forward process running through a wizard selecting source and target environments and the start replicating the VM cross the network, you can even have your virtual machines replicated to a Veeam Cloud & Service Provider, VCSP, if you don’t have a disaster recovery site of your own. The VCSP can have a hypervisor environment built for either Microsoft Hyper-V, VMware vSphere or VMware vCloud Director. VMware vCloud Director is VMware’s multi-tenant solution to host Infrastructure as a Service and purpose built specifically for Service Providers.

In this post I’m describing the process of replicating VMs to a VCSP using a feature of Veeam Backup & Replication called Cloud Connect, I’m not going through how to setup Cloud Connect. If you need more information about the ins and outs of Cloud Connect please visit Luca Dell’Oca’s webpage about Cloud Connect.

In the hosted environment at the VCSP you can power on virtual machines if needed to keep your business going if there’s a catastrophic event at your own site for instance a lengthy power outage, load shedding, you can even create a fail-over plan dictating which virtual machines should be powered on and in which order they should start, making sure everything starts in the correct order.

Replicating over the network may not be optimal in all scenarios, at least not the first initial full replication cycle. Let’s say you have a few very large virtual machines that you want to protect by sending them to a disaster recovery site hosted by your Veeam Cloud Service Provider but it’s too big to actually be transferred over the network within the available backup window, what do you do?

In Veeam Backup & Replication you can seed an initial copy of the virtual machine to your service provider using some sort of transportable solution. USB drives, Tapes or solutions of that nature – using “sneaker net”. The basic concept is to get a copy of the virtual machine to the service provider so they can import the VM to their environment and when you start replicating over the network you just send the changes made to the VM that has occurred since you made the copy of the VM. No need for a full transfer of the VM cross the network!

So the 3 basic steps that needs to be taken:

  • Backup VM to a transportable storage device and send it to VCSP
  • The VCSP imports the VM to the correct Org vDC in vCloud Director
  • Set up a replication job at the customer site using the imported VM at the VSCP site as mapping VM

If the service provider has a multi-tenant virtualization layer, meaning built on VMware vCloud Director, the process is simple but has to been broken down into a few distinct steps. If you as a service provider are using VMware vCloud Director 9.7, these are the steps you take if the customer has Veem Backup & Replication installed that can be used:

Step 1 – Customer environment
Backup source VM (normal backup job or VeeamZip) to a portable storage solution. Either backup to C:\Backup and move the backupfile manually to the USB device or select “VeeamZIP…” and specify the target USB devices directly.
VeeamZIP

Step 2 – Customer environment

Step 3 – Customer environment

Step 4 – Customer environment
When the backup is completed it should be visible in the “Backups”-section in “Disk (VeeamZIP)”

Step 5 – Customer environment
Transfer the backupfile using a transportable storage solution (a USB drive can be used)

Step 6 – VCSP environment
Connect USB drive and import backup file to Veeam Backup & Replication running at the VCSP data center. Click “Import Backup” in the top section.

Step 7 – VCSP environment
Select the backupfile on the USB device and click “Open” (you may need to change the file type selector to “Backup files (*.vbk)” to see the backupfile.

Step 8 – VCSP environment
Now Veeam Backup & Replication will import the backup

Step 9 – VCSP environment
Right click the VM from the imported backup and select “Restore entire VM…”

Step 10 – VCSP environment

Step 11 – VCSP environment
Select “Restore to a new location, or with different settings”

Step 12 – VCSP environment
Click “Host…”

Step 13 – VCSP environment
Select a host or a cluster that is under vCloud Director management where the customer has a virtual datacenter (shows up as a resource source pool in the next few steps)

Step 14 – VCSP environment
Select the VM and click “Pool…”

Step 15 – VCSP environment
Select resource pool (Org vDC of the customer)

Step 16 – VCSP environment

Step 17 – VCSP environment

Step 18 – VCSP environment
Map network adapter to desired network in the Org vDC

Step 19 – VCSP environment

Step 20 – VCSP environment

Step 21 – VCSP environment

Step 22 – VCSP environment
Log on to vCloud Director using the flex UI (the HTML5 UI lacks the “import from vSphere” option.

If you as a service provider are using VMware vCloud Director 10 with the new HTML5 UI for providers, please note that “import from vSphere” is not available in the H5 UI. What’s even more annoying is that the flex UI has also been disable by default in vCD 10 so to be able to import the VM into the Org vDC of the customer you first need to enable the flex UI of vCD:

Enable the vCloud Director Web Console

Step 23 – VCSP environment
Import VM in vCD from vSphere

Step 24 – VCSP environment
Select “Move VM” and not “Copy VM” in Import wizard

Step 25 – VCSP environment

Step 26 – VCSP environment

Step 27 – Customer environment
Set up a new replication job at customer side

Step 28 – Customer environment
Select “Replica seeding (for low bandwidth DR sites)”

Step 29 – Customer environment
Select the source VM from the customer production hypervisor (the same used in step 1)

Step 30 – Customer environment
In the “Destination”-tab, for the “Host or cluster:”-selection. Choose “Cloud host…”

Step 31 – Customer environment
Select the Org vDC to use (same as in step 15)

Step 32 – Customer environment
Select vApp and Storage policy to be used

Step 33 – Customer environment
Select desired restore points to keep

Step 34 – Customer environment
Select desired replication mode

Step 35 – Customer environment
In the “Seeding”-tab. In the “Replica mapping” section. Select “Map replicas to exsiting VMs”, click on the VM and select edit.

Step 36 – Customer environment
Select the seeded VM from step 17

Step 37 – Customer environment

Step 38 – Customer environment
Set a desired replication schedule

Step 39 – Customer environment
If desired: Click “Run the job when I click Finish”
Click “Finish”

Step 40 – Customer environment
Verify that replication successfully finish

Step 41 – Customer environment
The replication job only transfers changed blocks since the backup/import was made

How to setup Veeam replication with VMware vCloud Director

Veeam Backup & Replication 9.5 update 4 has now finally been released (to the VCSP community first and the general public on the 22:nd of January). There are loads of really interesting updates and new features.

Veeam Backup & Replication 9.5 update 4

To name a few of the enhancement/new features:

  • Capacity tier: Support for object based storage, gives you access to BLOB storage from Microsoft Azure, Amazon S3 and S3 compatible as well as IBM Cloud Object Storage. This is a new addition to Scale-Out backup repository users. You have your local “performance tier” as per usual but you can offload data based on age or space to object based storage.
  • Staged restore (GDPR compliance for instance, the right to be forgotten  or other use cases where you’d need to run a script on the VM before restoring it)
  • Secure restore where you can do a virus scan on the VM before restoring
  • Direct restore to Amazon EC2 – restoring to Azure has been available for a while but now you can also choose to restore your on-premises infrastructure VMs to Amazon EC2 – combined with the functionality of the backup vendor Veeam acquired a year ago called N2WS for backing up EC2 instances we now have a whole other level of portability of our data: backup everything, restore where it makes the most sense.
  • Self-service backup and restore portal using Enterprise manager
  • Enhancements to various Veeam explorers
  • Plugins for SAP HANA and Oracle RMAN
  • Platform support: vSphere 6.7 update 1, Windows Server 2019 and vCloud Director 9.5

But going back to the fact that update 4 now is available for VCSPs (or Veeam Cloud & Service Provider), there have been some updates for VMware environments as well (VMware calls their service provider program “VCPP”). Included in the VCPP program is a great product called vCloud Director that has been around for ages but is only available for service providers to use nowadays. VMware vCloud Director is an abstraction layer on top of vCenter so up until now there has been no support for vCloud Director for Veeam Cloud Connect usage when replicating VMs from a customer to the service provider environment. The solution previously was to replicate VMs to the service provider vCenter using Cloud Connect and then manually import VMs to the correct organization from vCloud Director. With update 4 that manual step has now been removed, and the process has in fact been improved since the customer can – using cloud connect and a single port mind you! (no VPN required) – replicate virtual machines from the onsite vSphere environment directly to their own Organization and Org vDC. The customer can also set up failover plans and run those if needed all using the same vCloud Director credentials they already received from the service provider.

It’s really easy to setup, below is a video where I show you how to configure the service provider bits such as adding vCloud Director, setting up tenants but also how the customer would configure their environment i.e. how to connect to a service provider using Cloud Connect and setting up replication jobs from a local environment and replicating VMs to the service provider vCloud Director and the customers org vDC within that environment.

(The video is in swedish but just turn off the sound if you don’t understand)

You’re missing out as a Service Provider if you’re not providing backups for Office 365

Hopefully you’ve already heard, Office 365 is a big hit for just about any vertical and customer type but have you had the much, much, needed conversation with your customers on the necessity of protecting the data that’s now landed in Office 365? I’ve said it before and I’ll say it again: Microsoft is fantastic in providing availability of the service they’re providing but however they also say that any data you store in Office 365 is yours – meaning you have the responsibility to actually think about how you’re going to protect that data and in the end also providing some sort of backup mechanism that executes the backups for you. This is described in a blog post from Veeam called the Office 365 shared responsibility model, which is an essential read if you haven’t already seen it.

A few months ago Veeam released the update version of Backup for Office 365, version 2.0,  and we’re now able to not only backup the mail part of Office 365 but also Sharepoint and Onedrive.

As a Service Provider, Veeam has a program called VCSP (Veeam Cloud & Service Provider), you have the ability to provide Backup as a Service and Disaster Recovery as a Service based on a specific Veeam Backup & Replication function called Cloud Connect available only to Service Providers. Now in relation to Office 365 you have the ability to leverage Cloud Connect to provide backup for Office 365 as a service as well for your customers. So if you are a service provider today, already using Cloud Connect – Why are you not providing backup for Officec 365 as a service? If you have Cloud Connect already installed it takes less than 10 minutes to set up the new service.

So how difficult is it to set up? Not difficult at all – in fact I’ll show you in the video below (Swedish only, but it’s not rocket science so if you’re not swedish speaking it should be fairly easy to follow along anyway). But it basically boils down to these 5 steps:

  1. Install Veeam Backup for Office 365
  2. Install a certificate
  3. Enable tenants authentication with organization credentials
  4. Configure a repository for the customer
  5. Add the customer account and set up a backup job

That’s it! In the video I will also show you how to set up a restore environment at the customer site that will let them restore items themselves using their administrative Office 365 credentials using a local installation of Backup & Replication Free edition and Veeam Explorers for Exchange and Sharepoint, but there are actually a few different ways of restoring – I’m just showing one of the options. You could also have the customers logging on to the Backup server itself for instance or provide a web portal to manage the retores. When restoring items, as always with Veeam, you have multiple destinations for your restore jobs; restore back to Office 365 (as shown in the video), restore to a .pst-file or restore an item and send it as an attachment to a mail to someone. But that’s not all, you can actually restore back to an on-premises installation of Microsoft Exchangeas well  if you’d like. In fact you can use Backup for Office 365 to do backups of your on-premises Exchange server so you have not only a backup tool but a migration tool as well – working bi-directional anyway you want!

Here’s the installation and configuration video! (Swedish only)

Veeam Backup & Replication update 3 released!

Update 3 has just been released for Veeam Backup & Replication. Update 3 comes with a lot of new cool functionality:

Storage Snapshots Integrations
New strategic partnerships and storage integrations include:

  • IBM Spectrum Virtualize – IBM SAN Volume Controller (SVC) and the IBM Storwize family
  • Lenovo Storage V series

Bringing functionalities to these arrays like Backup from Storage Snapshots, Explorers for Storage Snapshots

Recycle bin for Cloud Connect
As a way of protecting backups from maliciously being deleted if someone gets access to the tenants credentials, deleted backups will now be placed in a recycle bin for a configurable amount of days but to the user they’re gone. This way it will still be possible to retrieve backup files and restore VMs and/or files even though from a user perspective the backup files seems lost. Once the backup files have been deleted and temporarily placed in the recycle bin, the backup files will not consume valuable resources from the disk quota. When the deleted backup files needs to be used the tenant has to contact the service provider to have the service provider restore the backup files from the recycle bin back to the tenants repository.

VMware Cloud on AWS
Support for backing up VMs running och VMware Cloud on AWS

Veeam ONE
Agent monitoring and reporting

  • Protected agents
  • Agent backup status
  • Identify agents with no backup copy

Backup Compliance reporting

  • Geolocation of Data Protection Report: List all data sources grouped by production location and location of their copies/replicas
  • Data Geolocation Mismatch Report: List all data sources that have one or more copies where the location is different from the production data

Agent only use: 0-socket license required for enabling advanced funtionalities (Scale-Out Backup repository, Tape, WAN accelerator) in Backup & Replication when using agents if you’re not using Backup & Replication for backing up any virtual environment.

Veeam Agents
Centralized deployment and management giving you a single pane of glass for all backups and restores regardless of location in the environment – VMs or physical servers, you can even install Agents on VMs running in the Cloud or on any hypervisor.

Veeam Agents for Microsoft Windows 2.1

  • Windows Failover Cluster
    • Includes SQL AlwaysOn, Windows Failover Cluster and Exchange Database Availability Groups
  • Change Block tracking driver for faster incremental backups of Windows Servers
  • Microsoft OneDrive support as a backup target

This means you can you Veeam Explorers to restore application-items from Exchange, SQL

Veeam Agents for Linux 2.0

  • Scale-Out Backup Repository
  • Direct backup to Cloud Connect
  • Source side encryption

Release notes can be found here

In 5 minutes: Veeam Cloud Connect

Time again for a new episode in the “in 5 minutes” series. Today I’ll be discussing Veeam Cloud Connect.

Swedish only!