How to setup Veeam replication with VMware vCloud Director

Veeam Backup & Replication 9.5 update 4 has now finally been released (to the VCSP community first and the general public on the 22:nd of January). There are loads of really interesting updates and new features.

Veeam Backup & Replication 9.5 update 4

To name a few of the enhancement/new features:

  • Capacity tier: Support for object based storage, gives you access to BLOB storage from Microsoft Azure, Amazon S3 and S3 compatible as well as IBM Cloud Object Storage. This is a new addition to Scale-Out backup repository users. You have your local “performance tier” as per usual but you can offload data based on age or space to object based storage.
  • Staged restore (GDPR compliance for instance, the right to be forgotten  or other use cases where you’d need to run a script on the VM before restoring it)
  • Secure restore where you can do a virus scan on the VM before restoring
  • Direct restore to Amazon EC2 – restoring to Azure has been available for a while but now you can also choose to restore your on-premises infrastructure VMs to Amazon EC2 – combined with the functionality of the backup vendor Veeam acquired a year ago called N2WS for backing up EC2 instances we now have a whole other level of portability of our data: backup everything, restore where it makes the most sense.
  • Self-service backup and restore portal using Enterprise manager
  • Enhancements to various Veeam explorers
  • Plugins for SAP HANA and Oracle RMAN
  • Platform support: vSphere 6.7 update 1, Windows Server 2019 and vCloud Director 9.5

But going back to the fact that update 4 now is available for VCSPs (or Veeam Cloud & Service Provider), there have been some updates for VMware environments as well (VMware calls their service provider program “VCPP”). Included in the VCPP program is a great product called vCloud Director that has been around for ages but is only available for service providers to use nowadays. VMware vCloud Director is an abstraction layer on top of vCenter so up until now there has been no support for vCloud Director for Veeam Cloud Connect usage when replicating VMs from a customer to the service provider environment. The solution previously was to replicate VMs to the service provider vCenter using Cloud Connect and then manually import VMs to the correct organization from vCloud Director. With update 4 that manual step has now been removed, and the process has in fact been improved since the customer can – using cloud connect and a single port mind you! (no VPN required) – replicate virtual machines from the onsite vSphere environment directly to their own Organization and Org vDC. The customer can also set up failover plans and run those if needed all using the same vCloud Director credentials they already received from the service provider.

It’s really easy to setup, below is a video where I show you how to configure the service provider bits such as adding vCloud Director, setting up tenants but also how the customer would configure their environment i.e. how to connect to a service provider using Cloud Connect and setting up replication jobs from a local environment and replicating VMs to the service provider vCloud Director and the customers org vDC within that environment.

(The video is in swedish but just turn off the sound if you don’t understand)

You’re missing out as a Service Provider if you’re not providing backups for Office 365

Hopefully you’ve already heard, Office 365 is a big hit for just about any vertical and customer type but have you had the much, much, needed conversation with your customers on the necessity of protecting the data that’s now landed in Office 365? I’ve said it before and I’ll say it again: Microsoft is fantastic in providing availability of the service they’re providing but however they also say that any data you store in Office 365 is yours – meaning you have the responsibility to actually think about how you’re going to protect that data and in the end also providing some sort of backup mechanism that executes the backups for you. This is described in a blog post from Veeam called the Office 365 shared responsibility model, which is an essential read if you haven’t already seen it.

A few months ago Veeam released the update version of Backup for Office 365, version 2.0,  and we’re now able to not only backup the mail part of Office 365 but also Sharepoint and Onedrive.

As a Service Provider, Veeam has a program called VCSP (Veeam Cloud & Service Provider), you have the ability to provide Backup as a Service and Disaster Recovery as a Service based on a specific Veeam Backup & Replication function called Cloud Connect available only to Service Providers. Now in relation to Office 365 you have the ability to leverage Cloud Connect to provide backup for Office 365 as a service as well for your customers. So if you are a service provider today, already using Cloud Connect – Why are you not providing backup for Officec 365 as a service? If you have Cloud Connect already installed it takes less than 10 minutes to set up the new service.

So how difficult is it to set up? Not difficult at all – in fact I’ll show you in the video below (Swedish only, but it’s not rocket science so if you’re not swedish speaking it should be fairly easy to follow along anyway). But it basically boils down to these 5 steps:

  1. Install Veeam Backup for Office 365
  2. Install a certificate
  3. Enable tenants authentication with organization credentials
  4. Configure a repository for the customer
  5. Add the customer account and set up a backup job

That’s it! In the video I will also show you how to set up a restore environment at the customer site that will let them restore items themselves using their administrative Office 365 credentials using a local installation of Backup & Replication Free edition and Veeam Explorers for Exchange and Sharepoint, but there are actually a few different ways of restoring – I’m just showing one of the options. You could also have the customers logging on to the Backup server itself for instance or provide a web portal to manage the retores. When restoring items, as always with Veeam, you have multiple destinations for your restore jobs; restore back to Office 365 (as shown in the video), restore to a .pst-file or restore an item and send it as an attachment to a mail to someone. But that’s not all, you can actually restore back to an on-premises installation of Microsoft Exchangeas well  if you’d like. In fact you can use Backup for Office 365 to do backups of your on-premises Exchange server so you have not only a backup tool but a migration tool as well – working bi-directional anyway you want!

Here’s the installation and configuration video! (Swedish only)

Start protecting Office 365 in under 3 minutes

Office 365 is getting a lot of well deserved attention,  it’s an easy to use platform to provide your company with lots functionality without the need to heavily invest in on-premises infrastructure and hardware. Microsoft makes sure Office 365 is highly available and have all bits and pieces redundant. However, Microsoft does not own your data – You do! There are lots of different ways you might lose data: accidental deletion, ransomware and so on… A good read on the subject is a white paper from Veeam wich discusses 6 different areas:

  • Accidental deletion
  • Retention policy gaps and confusion
  • Internal security threats
  • External security threats
  • Legal and compliance requirements
  • Managing hybrid email deployments and migrations to Office 365

“With Office 365, it’s your data. You own it. You control it.”

-Microsoft

Since it’s your data it is also your responsibility to protect your data.  Now, how would you go about doing that?

Veeam Backup for Office 365 is currently in version 1.5 and supports backing up the mail environment of Office 365. Release 2.0 has been announced and will be released sometime later this year (at VeeamON maybe?), with version 2.0 you will be able to also backup OneDrive and SharePoint. And if you buy Veeam Backup for Office 365 today (version 1.5) you will get the added features down the line  with no additional fees or purchases, just upgrade the installation with the latest bits. Great stuff!

Another cool feature of Office 365 / Veeam Backup for Office 365 is that since both Office 365 and an on-premises Exchange server uses the same APIs you can use Veeam Backup for Office 365 to back up both environments if you have a hybrid installation, or you can even use Veeam as a migration tool – unidirectional of course!

But what does it take to start protecting your Office 365 mail environment? As it turns out, not a lot! It is REALLY simple to start backing up. As the video will show you, it took me no more than 3 minutes to start the first backup, quite impressive and again so simple to install.

See for yourself, the video shows installation of both Veeam Backup for Office 365 and Veeam Explorer for Exchange and configuring a new backup job in under 3 minutes:

PowerShell for the win!

Who doesn’t love PowerShell and PoweCLI? I use it to automate as much as I can. Building demo environments, upgrading stuff or just playing around. The list is just a few examples of available modules from the Microsoft PowerShell Gallery, you can spend hours exploring interesting modules there. The list below is mostly a reminder for myself but feel free to explore!

VMware

vSphere

https://www.powershellgallery.com/packages/Vester/

https://www.powershellgallery.com/packages/vDocumentation/

https://www.powershellgallery.com/packages/Get-VMotion/

https://www.powershellgallery.com/packages/createsnapshots/

https://www.powershellgallery.com/packages/VMW_RemoveOldSnapshots/

https://www.powershellgallery.com/packages/Virten.net.VimAutomation/

https://www.powershellgallery.com/packages/install-vmwworkstation/

 

vRA

https://www.powershellgallery.com/packages/UMN-VMWareRA/

 

vCloud

https://www.powershellgallery.com/packages/Invoke-vCloud/

 

NSX

https://www.powershellgallery.com/packages/PowerNSX/

https://www.powershellgallery.com/packages/TestNBPNSX/

 

Log Insight

https://www.powershellgallery.com/packages/LogInsight/

Veeam

https://www.powershellgallery.com/packages/Get-VeeamBackupReport/

https://www.powershellgallery.com/packages/Set-VeeamBackupWindowOptions/

 

Pure Storage

https://www.powershellgallery.com/packages/PureStoragePowerShellToolkit/

https://www.powershellgallery.com/packages/PureStoragePowerShellSDK/

Amazon Web Services

https://www.powershellgallery.com/packages/AWSPowerShell/

https://www.powershellgallery.com/packages/AWSPowerShell.NetCore/

Dell

https://www.powershellgallery.com/packages/DellBIOSProvider/

https://www.powershellgallery.com/packages/DellWarranty/

Övrigt

https://www.powershellgallery.com/packages/WinSCP/

https://www.powershellgallery.com/packages/SSH/

https://www.powershellgallery.com/packages/Posh-SSH/

https://www.powershellgallery.com/packages/TreeSize/

https://www.powershellgallery.com/packages/PowershellBGInfo/

https://www.powershellgallery.com/packages/Tesla/

https://www.powershellgallery.com/packages/Ravello/

https://www.powershellgallery.com/packages/Telldus/

https://www.powershellgallery.com/packages/PSSpotify/

Veeam Backup & Replication update 3 released!

Update 3 has just been released for Veeam Backup & Replication. Update 3 comes with a lot of new cool functionality:

Storage Snapshots Integrations
New strategic partnerships and storage integrations include:

  • IBM Spectrum Virtualize – IBM SAN Volume Controller (SVC) and the IBM Storwize family
  • Lenovo Storage V series

Bringing functionalities to these arrays like Backup from Storage Snapshots, Explorers for Storage Snapshots

Recycle bin for Cloud Connect
As a way of protecting backups from maliciously being deleted if someone gets access to the tenants credentials, deleted backups will now be placed in a recycle bin for a configurable amount of days but to the user they’re gone. This way it will still be possible to retrieve backup files and restore VMs and/or files even though from a user perspective the backup files seems lost. Once the backup files have been deleted and temporarily placed in the recycle bin, the backup files will not consume valuable resources from the disk quota. When the deleted backup files needs to be used the tenant has to contact the service provider to have the service provider restore the backup files from the recycle bin back to the tenants repository.

VMware Cloud on AWS
Support for backing up VMs running och VMware Cloud on AWS

Veeam ONE
Agent monitoring and reporting

  • Protected agents
  • Agent backup status
  • Identify agents with no backup copy

Backup Compliance reporting

  • Geolocation of Data Protection Report: List all data sources grouped by production location and location of their copies/replicas
  • Data Geolocation Mismatch Report: List all data sources that have one or more copies where the location is different from the production data

Agent only use: 0-socket license required for enabling advanced funtionalities (Scale-Out Backup repository, Tape, WAN accelerator) in Backup & Replication when using agents if you’re not using Backup & Replication for backing up any virtual environment.

Veeam Agents
Centralized deployment and management giving you a single pane of glass for all backups and restores regardless of location in the environment – VMs or physical servers, you can even install Agents on VMs running in the Cloud or on any hypervisor.

Veeam Agents for Microsoft Windows 2.1

  • Windows Failover Cluster
    • Includes SQL AlwaysOn, Windows Failover Cluster and Exchange Database Availability Groups
  • Change Block tracking driver for faster incremental backups of Windows Servers
  • Microsoft OneDrive support as a backup target

This means you can you Veeam Explorers to restore application-items from Exchange, SQL

Veeam Agents for Linux 2.0

  • Scale-Out Backup Repository
  • Direct backup to Cloud Connect
  • Source side encryption

Release notes can be found here

How and why series – Veeam Backup & Replication SureBackup

After the first episode of my “How and why”-series in which I talked about VMware VSAN I thought it’d be fun to show you why I love Veeam so much and in particular the function SureBackup. SureBackup is all about verifying your backup data (your VMs) in a super smart way. The bottom line is, if you need to restore anything you KNOW that the restore will work.

Again, the video is in swedish only.

(Re)claim your space!

In one of my previous posts, Is Bitlooker from Veeam a game-changer?, I wrote about the benefits of using Bitlooker for backup jobs when using Veeam Backup & Replication v9.x however Bitlooker is a feature that is not only available for backup jobs – you can use it for replication jobs as well.

So I thought it’d be fun to see what difference, if any, it makes. The goal of my tests is to figure out the most effective way of copying/replicating a VM from one host to another.

The set up for the test:

A virtual machine is installed with Windows Server 2016 standard edition. 100 GB disk assigned to the VM, thin provisioned. The disk is then filled with files (a bunch of iso-files of different sizes). That’s the baseline, then roughly 85 GB will be deleted (all the added iso-files) – then trashcan will be emptied. So we’ll have some blocks containing stale/old data, the blocks are marked as available to be reused from the operating system point of view but they haven’t been zeroed out so from any hypervisor (outside the VM) it just looks as any other block containing data.

Operating system installed (Windows Server 2016) and updated. The VM now consumes 13,5 GB worth of storage.

Then a bunch of files were added (almost) filling the entire disk.

From the vSphere side of it:

At this point the just added files were removed and trashcan emptied.

And from vSphere:

Now the command ‘ls’ will not show the actual size, so ‘du’ can be used instead to see the actual size of the vmdk file:

I’m going to test 4 different scenarios:

  1.  Migrate the VM from one host to another offline (VM will be shutdown).
  2. Replicating the virtual machine with VMware vSphere Replication 6.5.
  3. Replicating the virtual machine with Veeam Backup & Replication without Bitlooker enabled.
  4. Replicating the virtual machine with Veeam Bitlooker enabled.

The thesis, or point to prove, is that test 1-3 will have no or little impact on the size of the vmdk file however – magic will happen on test 4. So lets perform the tests and find out for real!

Test 1:

VM moved to another host while offline and now let’s explore what can be seen using different methods.

Inside the VM:

From the host:

So no change in vmdk file size as expected.

Test 2:

The virtual machine will be replicated to another host using VMware vSphere Replication 6.5.

VMware vSphere has been configured using the following settings:

Not alot of settings, in fact, the above settings will have no impact on the vmdk size. They will only have control how the snapshot on the VM will be generated (crash consistent vs consistent backup) and the impact the replication job will have on the network.

Inside the VM:

From the host:

Since ‘ls’ doesn’t show the actual size on a thin disk, disk usage ‘du’ is used instead:

So no change in vmdk file size as expected.

Test 3:

The virtual machine is replicated to another host using Veeam Backup & Replication v9.5.

Replication from a Veeam perspective has been set up, to make a fair comparison to the VMware replication test (test 2), the Veeam job will not use exclude swap file blocks:

Processed and read data in the picture below tells us that Veeam doesn’t know the difference between blocks in use and blocks marked as deleted (the same applies for almost all backup vendors):

Inside the VM:

From the host:

And using ‘du’:

So no change in vmdk file size as expected.

 

Test 4:

Know time for the fun stuff. The virtual machine will be replicated to another host using Veeam Backup & Replication v9.5. We will use both space saving techniques we can enable on the job (with application-aware processing we can also exclude specific files, folders, file extensions but we’re not using that feature in this test)

Now, this is the magic we were looking for! The  proxy server has processed all of the data but it has only read data that contain used blocks!

From the VM:

From the host:

Now the vSphere web client combine the .vmdk and -flat.vmdk file into one (like it’s done forever):

And the disk usage utility shows:

Yikes! That’s cool stuff!

Conclusion:

Bitlooker is a feature you should have enabled on any relevant job. It certainly can be used to reclaim that precious storage space you so desperately need.  Heck, why no use it as part of your normal failover testing, cause you’ll already doing that right? Once a month (or how often you feel appropriate) do a planned failover using Veeam Backup & Replication, verify that you DR plan works and as an added bonus you reclaim disk space in the process!

And yet another benefit is the spent replicating the virtual machine, without Bitlooker it took 30 minutes to replicate the VM from one host to another but it was just shy of 7 minutes with Bitlooker enabled.

So seriously, why are you not using this magic thing? There’s only one drawback, Bitlooker supports only NTFS file system (=Windows VMs).