Using AZURE_STORAGE_ENDPOINT_BASE together with BUCKET_STORAGE_MULTIPLE_ENDPOINTS allows you to configure buckets in multiple bucket services, for instance to migrate from hosted bucket storage to a local bucket storage service.

For more information, see AZURE_STORAGE_ENDPOINT_BASE.

You can point to your own hosting endpoint for Azure to use for bucket storage if you host an Azure-compatible service.

https://my-bucket.my-own-azure:8080/path/inside/bucket/file.txt

AZURE_STORAGE_ENDPOINT_BASE=http://{bucket}.my-own-azure:8080

If LogScale is set up to use an HTTP proxy, it will be used for communicating with Azure by default. To disable it, set:

# Use the globally configured HTTP proxy for communicating with Azure.
  # Default is true.
  AZURE_STORAGE_USE_HTTP_PROXY=false

The following configurations allow tuning for performance. There may be a cost associated with increasing these variables as Azure is billed based on the number of operations executed.

# How many parallel chunks to split each file into when uploading and downloading.
  # Defaults to 4, the maximum is 16.
  AZURE_STORAGE_CHUNK_COUNT=4

  # Maximum number of files that LogScale will run concurrent downloads or uploads for at a time.
  # Default is the number of hyperthreads / 2
  AZURE_STORAGE_CONCURRENCY=10

  # Chunk size for uploads and download ranges. Max 8 MB, which is the default.
  # Minimum value is 5 MB.
  AZURE_STORAGE_CHUNK_SIZE=8388608

  # Prefer to fetch data files from the bucket when possible even if another
  # node in the LogScale cluster also has a copy of the file.
  # In some environments, it may be less expensive to do the transfers this way.
  # The transfer from the bucket may be billed at a lower cost than a transfer from
  # a node in another region or in another data center.  This preference does not
  # guarantee that the bucket copy will be used, as the cluster can
  # still make internal replicas directly in case the file is not yet in
  # a bucket.
  # Default is false.
  AZURE_STORAGE_PREFERRED_COPY_SOURCE=false

By default, once uploaded to Azure, files are not overwritten. LogScale uses an If-None-Match header, so any attempt at additional upload to a key that already exists in a bucket will fail prior to any file data bytes actually being sent. You might also see azure.statusCode() == 409. Status code 409 means that the file already exists in the bucket and the LogScale If-None-Match header is in place, preventing overwrites of files.