com.facebook.buck.artifact_cache

Class TwoLevelArtifactCacheDecorator

java.lang.Object

com.facebook.buck.artifact_cache.TwoLevelArtifactCacheDecorator

All Implemented Interfaces:

ArtifactCache, CacheDecorator, AutoCloseable

public class TwoLevelArtifactCacheDecorator
extends Object
implements ArtifactCache, CacheDecorator

The DirArtifactCache and HttpArtifactCache caches use a straightforward rulekey -> (metadata, artifact) mapping. This works well and is easy to reason about. That also means that we will fetch `artifact` whenever they `key` or `metadata` change possibly resulting in fetching the same artifact multiple times. This class is an attempt at reducing the number of repeated fetches for the same artifact. The data is stored and retrieved using the following scheme: rulekey -> (metadata, content hash) content hash -> artifact This means we only download the artifact when its contents change. This means that rules with different keys but identical outputs require less network bandwidth at the expense of doubling latency for downloading rules whose outputs we had not yet seen.

Constructor Summary

Constructors

Constructor and Description
TwoLevelArtifactCacheDecorator(ArtifactCache delegate, ProjectFilesystem projectFilesystem, BuckEventBus buckEventBus, boolean performTwoLevelStores, long minimumTwoLevelStoredArtifactSize, Optional<Long> maximumTwoLevelStoredArtifactSize)

Method Summary

Modifier and Type	Method and Description
void	close()
com.google.common.util.concurrent.ListenableFuture<CacheDeleteResult>	deleteAsync(List<RuleKey> ruleKeys)
com.google.common.util.concurrent.ListenableFuture<CacheResult>	fetchAsync(BuildTarget target, RuleKey ruleKey, LazyPath output) Fetch a cached artifact, keyed by ruleKey, save the artifact to path specified by output, and return true on success.
CacheReadMode	getCacheReadMode() This method must return the same value over the lifetime of this object.
ArtifactCache	getDelegate()
com.google.common.util.concurrent.ListenableFuture<com.google.common.collect.ImmutableMap<RuleKey,CacheResult>>	multiContainsAsync(com.google.common.collect.ImmutableSet<RuleKey> ruleKeys) Contains is supposed to be best-effort, but super-fast => Assume the second level is present.
void	skipPendingAndFutureAsyncFetches() All pending (and future) async fetches will be immediately marked as skipped.
com.google.common.util.concurrent.ListenableFuture<Unit>	store(ArtifactInfo info, BorrowablePath output) Store the artifact at path specified by output to cache, such that it can later be fetched using ruleKey as the lookup key.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.facebook.buck.artifact_cache.ArtifactCache

store

Constructor Detail

TwoLevelArtifactCacheDecorator

public TwoLevelArtifactCacheDecorator(ArtifactCache delegate,
                                      ProjectFilesystem projectFilesystem,
                                      BuckEventBus buckEventBus,
                                      boolean performTwoLevelStores,
                                      long minimumTwoLevelStoredArtifactSize,
                                      Optional<Long> maximumTwoLevelStoredArtifactSize)

Method Detail

fetchAsync

public com.google.common.util.concurrent.ListenableFuture<CacheResult> fetchAsync(@Nullable
                                                                                  BuildTarget target,
                                                                                  RuleKey ruleKey,
                                                                                  LazyPath output)

Description copied from interface: ArtifactCache

Fetch a cached artifact, keyed by ruleKey, save the artifact to path specified by output, and return true on success.

Specified by:

fetchAsync in interface ArtifactCache

Parameters:

target - rule for which this is an artifact

ruleKey - cache fetch key

output - Path to store artifact to. Path should not be accessed unless store operation is guaranteed by the cache, to avoid potential extra disk I/O.

Returns:

whether it was a CacheResultType.MISS (indicating a failure) or some type of hit.

skipPendingAndFutureAsyncFetches

public void skipPendingAndFutureAsyncFetches()

Description copied from interface: ArtifactCache

All pending (and future) async fetches will be immediately marked as skipped.

Specified by:

skipPendingAndFutureAsyncFetches in interface ArtifactCache

store

public com.google.common.util.concurrent.ListenableFuture<Unit> store(ArtifactInfo info,
                                                                      BorrowablePath output)

Description copied from interface: ArtifactCache

Store the artifact at path specified by output to cache, such that it can later be fetched using ruleKey as the lookup key. If any internal errors occur, fail silently and continue execution. Store may be performed synchronously or asynchronously.

This is a noop if ArtifactCache.getCacheReadMode()} returns READONLY.

Specified by:

store in interface ArtifactCache

Parameters:

info - information to store with the artifact

output - path to read artifact from. If its borrowable, you may freely move the file into cache without obtaining a copy of the file.

Returns:

ListenableFuture that completes once the store has finished.

multiContainsAsync

public com.google.common.util.concurrent.ListenableFuture<com.google.common.collect.ImmutableMap<RuleKey,CacheResult>> multiContainsAsync(com.google.common.collect.ImmutableSet<RuleKey> ruleKeys)

Contains is supposed to be best-effort, but super-fast => Assume the second level is present.

Specified by:

multiContainsAsync in interface ArtifactCache

Parameters:

ruleKeys - Set of cache fetch keys.

Returns:

map of keys to CacheResult which can be a CacheResultType.MISS / CacheResultType.ERROR (indicating a failure) or CacheResultType.CONTAINS.

deleteAsync

public com.google.common.util.concurrent.ListenableFuture<CacheDeleteResult> deleteAsync(List<RuleKey> ruleKeys)

Specified by:

deleteAsync in interface ArtifactCache

getDelegate

public ArtifactCache getDelegate()

Specified by:

getDelegate in interface CacheDecorator

Returns:

Cache that is wrapped by a decorator.

getCacheReadMode

public CacheReadMode getCacheReadMode()

Description copied from interface: ArtifactCache

This method must return the same value over the lifetime of this object.

Specified by:

getCacheReadMode in interface ArtifactCache

Returns:

whether thisArtifactCache supports storing artifacts.

close

public void close()

Specified by:

close in interface ArtifactCache

Specified by:

close in interface AutoCloseable