https://docs.openstack.org/swift/rocky/overview_large_objects.html
Dynamic Large Objects
:: 5GB 를 초과하는 Objects는 5GB 이하의 세그먼트로 분할하여 업로드
시나리오
100M 파일을 만들어 업로드 / 다운로드
# dd if=/dev/zero of=test.img bs=1M count=100
# split -b 40000000 file
# ls -l
-rw-r--r-- 1 root root 104857600 10월 11 17:13 file
-rw-r--r-- 1 root root 40000000 10월 11 17:14 xaa
-rw-r--r-- 1 root root 40000000 10월 11 17:14 xab
-rw-r--r-- 1 root root 24857600 10월 11 17:14 xac
:: segments 파일 업로드
PUT https://{STORAGE_URL}/v1/{Container}/{Object}/{Object_Count} -H "X-Auth-Token: {Token}"
# curl -i -X PUT -H "X-Auth-Token: $(openstack token issue -c id -f value)" http://115.68.238.26/swift/v1/container3/file/001 --data-binary @xaa
HTTP/1.1 100 CONTINUE
HTTP/1.1 201 Created
etag: 48e9a108a3ec623652e7988af2f88867
Last-Modified: Fri, 11 Oct 2019 08:40:04 GMT
X-Trans-Id: tx000000000000000120d37-005da03fe3-1ca71b-default
X-Openstack-Request-Id: tx000000000000000120d37-005da03fe3-1ca71b-default
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Date: Fri, 11 Oct 2019 08:40:04 GMT
# curl -i -X PUT -H "X-Auth-Token: $(openstack token issue -c id -f value)" http://115.68.238.26/swift/v1/container3/file/002 --data-binary @xab
# curl -i -X PUT -H "X-Auth-Token: $(openstack token issue -c id -f value)" http://115.68.238.26/swift/v1/container3/file/003 --data-binary @xac
manifest 개체 업로드
- 모든 Objects 세그먼트를 업로드한 이후에 매니패스트 개체를 생성하게되면 하나의 개체처럼 사용할수 있습니다.
매니패스트 개체는 세그먼트들이 저장된 경로를 가리킨다.
PUT https://{STORAGE_URL}/v1/{Container}/{Object} -H "X-Objects-Manifest: {Container}/{Object}/" -H "X-Auth-Token: {Token}"
# curl -i -X PUT -H "X-Auth-Token: $(openstack token issue -c id -f value)" -H 'X-Object-Manifest: container3/file/' http://115.68.238.26/swift/v1/container3/file --data-binary ''
HTTP/1.1 201 Created
etag: d41d8cd98f00b204e9800998ecf8427e
Last-Modified: Fri, 11 Oct 2019 08:41:12 GMT
X-Trans-Id: tx000000000000000120d85-005da04027-1ca71b-default
X-Openstack-Request-Id: tx000000000000000120d85-005da04027-1ca71b-default
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Date: Fri, 11 Oct 2019 08:41:12 GMT
확인
# swift list container3
file
file/001
file/002
file/003
# swift stat container3 file
Account: v1
Container: container3
Object: file
Content Type: application/x-www-form-urlencoded
Content Length: 104857600
Last Modified: Fri, 11 Oct 2019 08:41:12 GMT
ETag: "4272d3e7c028e7f2b8be1ee97bf63864"
Manifest: container3/file/
Accept-Ranges: bytes
X-Timestamp: 1570783272.03245
X-Trans-Id: tx000000000000000120fd3-005da04272-1ca71b-default
X-Openstack-Request-Id: tx000000000000000120fd3-005da04272-1ca71b-default
# swift stat container3 file/001
Account: v1
Container: container3
Object: file/001
Content Type: application/x-www-form-urlencoded
Content Length: 40000000
Last Modified: Fri, 11 Oct 2019 08:40:04 GMT
ETag: 48e9a108a3ec623652e7988af2f88867
Accept-Ranges: bytes
X-Timestamp: 1570783204.51777
X-Trans-Id: tx000000000000000120fdd-005da04279-1ca71b-default
X-Openstack-Request-Id: tx000000000000000120fdd-005da04279-1ca71b-default
Download
# curl -H "X-Auth-Token: $(openstack token issue -c id -f value)" http://115.68.238.26/swift/v1/container3/file -O
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 100M 100 100M 0 0 52.0M 0 0:00:01 0:00:01 --:--:-- 52.0M
# ls
file xaa xab xac
# ls -alh
total 201M
drwxr-xr-x 2 root root 4.0K 10월 11 17:54 .
drwxr-xr-x 3 root root 552K 10월 11 17:31 ..
-rw-r--r-- 1 root root 100M 10월 11 17:54 file
-rw-r--r-- 1 root root 39M 10월 11 17:14 xaa
-rw-r--r-- 1 root root 39M 10월 11 17:14 xab
-rw-r--r-- 1 root root 24M 10월 11 17:14 xac
# swift delete container3 file
container3/file/003
container3/file/002
container3/file/001
file
