com.facebook.buck.remoteexecution.util

Class MultiThreadedBlobUploader

java.lang.Object

com.facebook.buck.remoteexecution.util.MultiThreadedBlobUploader

public class MultiThreadedBlobUploader
extends Object

A simple multi-threaded blob uploader for uploading inputs/outputs to the CAS.

Before uploading a file, the uploader will check if the CAS already contains it.

All upload requests get added to a queue for the "missing check". Work threads will pull up to missingCheckLimit items off this queue and send a request to the CAS to find which it does/doesn't contain. Any that are missing will be added to a queue to be uploaded. Work threads will then pull those off and upload them. When the upload is finished, the future for that digest will be fulfilled.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MultiThreadedBlobUploader.CorruptArtifactException An exception that indicates that an upload failed because the artifact was corrupted.

Constructor Summary

Constructors

Constructor and Description
MultiThreadedBlobUploader(int missingCheckLimit, int uploadSizeLimit, ExecutorService uploadService, CasBlobUploader delegate)

Method Summary

Modifier and Type	Method and Description
com.google.common.util.concurrent.ListenableFuture<Unit>	addMissing(java.util.stream.Stream<UploadDataSupplier> dataSupplier) Uploads missing items to the CAS.
boolean	containsDigest(Protocol.Digest digest)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiThreadedBlobUploader

public MultiThreadedBlobUploader(int missingCheckLimit,
                                 int uploadSizeLimit,
                                 ExecutorService uploadService,
                                 CasBlobUploader delegate)

Method Detail

containsDigest

public boolean containsDigest(Protocol.Digest digest)

addMissing

public com.google.common.util.concurrent.ListenableFuture<Unit> addMissing(java.util.stream.Stream<UploadDataSupplier> dataSupplier)

Uploads missing items to the CAS.