public class ConcurrentProjectBuildFileParser extends Object implements ProjectBuildFileParser
ProjectBuildFileParserby creating and maintaining a pool of
PythonDslProjectBuildFileParser wrapper creates new instances of delegated parser
using a provided factory and adds them to worker pool. If free parser is available at the time a
request is made, it is reused, if not then it is recreated. Once parsing request (aka
FileParser.getManifest(Path) is complete, parser is returned to the worker pool.
Worker pool of parsers can grow unconditionally so it is really up to the user of this class to
manage concurrency level by calling this class' methods appropriate number of times in parallel.
close() are not synchronized with the worker pool and just call
appropriate methods from all the parsers created to the moment. They should only be called when
no parsing is in effect.
The main reason for this class is to overcome limitations of current implementation of
PythonDslProjectBuildFileParser which is stateful and not multithreaded, but Graph Engine
transformations require parsers to be concurrent.
Another option would be to modify existing
PythonDslProjectBuildFileParser to be
concurrent. However the legacy
Parser pipelines use
ResourcePools to shard parser instances, though in fact it is
not needed for
SkylarkProjectBuildFileParser which is
concurrent. So, it will require to refactor existing parser pipelines to get rid of
ResourcePool which is a bigger change, so it probably should
be done once parsing is fully switched to Graph Engine.
|Constructor and Description|
Create new instance of
|Modifier and Type||Method and Description|
Collects the loaded files and extensions when parsing the
Collect all information from a particular, along with metadata about the information, for example which other files were also parsed.
Checks if existing
Reports profile information captured while parsing build files.
public ConcurrentProjectBuildFileParser(java.util.function.Supplier<ProjectBuildFileParser> projectBuildFileParserFactory)
projectBuildFileParserFactory- Factory that will be used for creating new instances of
ProjectBuildFileParseron demand. In order to make wrapped
ProjectBuildFileParserthread-safe this factory should create instances for each invocation, not memoize them, otherwise the same object will be used for all threads.
public BuildFileManifest getManifest(Path buildFile) throws BuildFileParseException, InterruptedException, IOException
public void reportProfile() throws IOException
public com.google.common.collect.ImmutableSortedSet<String> getIncludedFiles(Path buildFile) throws BuildFileParseException, InterruptedException, IOException
public boolean globResultsMatchCurrentState(Path buildFile, com.google.common.collect.ImmutableList<GlobSpecWithResult> existingGlobsWithResults) throws IOException, InterruptedException
GlobSpecs with results are the same as current state in the file system.
buildFile- the buildFile location to be used by the Globber.
existingGlobsWithResults- the existing (deserialized)
GlobSpecWithResultto check the file system state against.
trueif glob expansion produces results matching previously recorded ones,