Jump to content
Welcome to our new Citrix community!
  • 0

Resize VDI and LV larger than 2Tb problem

Vincent Couchouron


Hello there,


we'd to extend a virtual disk mounted on a Linux Debian VM. This VDI is only used for files sharing, so we can mount/umount it.


The issue is we need a VDI size larger than the 2tb Xenserver limitation.


Previous state: the VDI named "Data" is 2000Go size.


The SR size is 4,5To, so there's enough space to extend the VDI.


XenCenter did not allowed us to extend the VDI larger than 2To due to Xen limitation, so we extend it via lvresize command:

lvresize --config global{metadata_read_only=0} -l +100%FREE /dev/VG_XenStorage-325d812d-0f49-b8e4-a818-a221e5b54943/VHD-dab7b09f-f163-491d-b34d-8d8fb130e2cc


and the system answer "The VL has been resized successfully"


But, in XenCenter, the virtual disk still display the OLD size and when we re-attach the disk to the VM, it always shows the old size too.


We think there's a command to tell Xen to "refresh VDI size" or something like that.


Many thanks for your help.


Best regards


Link to comment

16 answers to this question

Recommended Posts

  • 1

Regarding this topic...


In a new Xen installation (8.2) we have a RAID1 (from xcp-ng) local SR of 14Tb. I've created, from command line, a 3Tb volume directly against the VG.


# lvcreate -L3T -nLV-dc2d998a-d156-4021-86e4-041d3ca61308 VG_XenStorage-95918c57-fa15-238b-62e2-a2ccf525acc5
  Device read short 82432 bytes remaining
  Logical volume "LV-dc2d998a-d156-4021-86e4-041d3ca61308" created.


Attached it to a VM, formatted it, mounted it, wrote some data on it. Then unmount, disconnect the volume from the VM. Following, lvresize adding another 1Tb


# lvresize -L+1T /dev/VG_XenStorage-95918c57-fa15-238b-62e2-a2ccf525acc5/LV-dc2d998a-d156-4021-86e4-041d3ca61308 
  Device read short 82432 bytes remaining
  Size of logical volume VG_XenStorage-95918c57-fa15-238b-62e2-a2ccf525acc5/LV-dc2d998a-d156-4021-86e4-041d3ca61308 changed from 3.00 TiB (786432 extents) to 4.00 TiB (1048576 extents).
  Logical volume VG_XenStorage-95918c57-fa15-238b-62e2-a2ccf525acc5/LV-dc2d998a-d156-4021-86e4-041d3ca61308 successfully resized.


Connected the volume again, checked that as expected the volume is now 4Tb in /proc/partitions


# cat /proc/partitions 
major minor  #blocks  name

 202        0   15728640 xvda
 202        1   15726592 xvda1
  11        0    1048575 sr0
 202       16 4294967296 xvdb


, and successfully extended the filesystem (from inside the VM of course, not as OP tried to do doing it against the LVM volume from Xen's cli).


# xfs_growfs /mnt/   
meta-data=/dev/xvdb              isize=512    agcount=4, agsize=201326592 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=0
data     =                       bsize=4096   blocks=805306368, imaxpct=5
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=393216, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 805306368 to 1073741824


Data still there of course :)


# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            464M     0  464M   0% /dev
tmpfs            96M   12M   85M  12% /run
/dev/xvda1       15G  1.3G   14G   9% /
tmpfs           479M     0  479M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           479M     0  479M   0% /sys/fs/cgroup
tmpfs            96M     0   96M   0% /run/user/0
/dev/xvdb       4.0T  4.7G  4.0T   1% /mnt


Then I've filled the disk (to be sure, using a command to actually write data on disk, not trickery like fallocate, dd+seek, etc) over 2Tb without problems. Using the "yes" command to create several big files, until over 2Tb were in use and everything is still working:


# yes zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz > largefile
# yes zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz > largefile2

# ll
total 2418916700
drwxr-xr-x  2 root root          149 Dec 19 20:40 ./
drwxr-xr-x 18 root root         4096 Dec  7 20:19 ../
-rw-r--r--  1 root root 202438533120 Dec 19 17:32 largefile
-rw-r--r--  1 root root 274931122176 Dec 19 18:02 largefile2
-rw-r--r--  1 root root 511587041280 Dec 19 18:46 largefile3
-rw-r--r--  1 root root 354791713991 Dec 19 19:16 largefile4
-rw-r--r--  1 root root 302649012887 Dec 19 19:42 largefile5
-rw-r--r--  1 root root 371127062254 Dec 19 20:14 largefile6
-rw-r--r--  1 root root 291943108608 Dec 19 20:40 largefile7
-rw-r--r--  1 root root 166815473664 Dec 19 20:55 largefile8

# df -Th .
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/xvdb      xfs   4.0T  2.3T  1.8T  57% /mnt



On 3/23/2018 at 4:41 PM, Mark Syms said:

Because the LV is only one layer in the storage. Within the LV it is formatted as a VHD to allow for having a hierarchy of parents and children (i.e. snapshots) and it is this internal VHD format that is limited to 2TB even if the containing volume is made much larger than this.


So at this point I don't know if some changes have been made to Xen because what Mark said doesn't seem to apply. I have a working volume bigger than 2Tb, that can be extended, and just works. 



Edited by jrubenc
Link to comment
  • 0

Tobias: the VDI contains 1,7To of files and works normally on the VM, so I did not run fsck... I'll test it!


chandrikas: we've already created some VDI sized of 4To and larger, and this works with command lines. Problem here is the resize...


for both: We know about the limitation and non official support, but we really need this... ;-) And we've already made this process, but I can't remember the commands :-(

Link to comment
  • 0

As Chandrika says above the 2TB is a hard limit imposed by the VHD image format. You can't make this VDI any bigger than 2 TB. If you need more than 2TB in the guest then you will need to attach multiple VDIs to the guest and either split the data inside the guest or use in guest volume spanning (mdadm RAID or LVM striping).



XenServer Storage Engineering.

Link to comment
  • 0

Because the LV is only one layer in the storage. Within the LV it is formatted as a VHD to allow for having a hierarchy of parents and children (i.e. snapshots) and it is this internal VHD format that is limited to 2TB even if the containing volume is made much larger than this.

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Create New...