mycelyso.pilyso.misc package¶

Submodules¶

mycelyso.pilyso.misc.h5writer module¶

The h5writer submodule contains the hdf5_output function, which accepts various datatypes (binary, image, tabular), and adds them to a (possibly pre-existing) HDF5 file.

class mycelyso.pilyso.misc.h5writer.CompressedObject(data, compression=None)[source]¶

Bases: object

compression¶

compressions = {'bz2': (<function CompressedObject.<lambda>>, <function CompressedObject.<lambda>>), 'lzma': (<function CompressedObject.<lambda>>, <function CompressedObject.<lambda>>), 'zlib': (<function CompressedObject.<lambda>>, <function CompressedObject.<lambda>>)}¶

data¶

debug = False¶

default_compression = 'bz2'¶

get()[source]¶

mycelyso.pilyso.misc.h5writer.acquire_lock(lock_file)[source]¶

mycelyso.pilyso.misc.h5writer.hdf5_node_name(s)[source]¶

mycelyso.pilyso.misc.h5writer.hdf5_output(_filename, immediate_prefix='', tabular_name='result_table')[source]¶

mycelyso.pilyso.misc.h5writer.release_lock(lock_file)[source]¶

mycelyso.pilyso.misc.h5writer.return_or_uncompress(something)[source]¶

mycelyso.pilyso.misc.h5writer.wait_for_lock_and_prepare_filename(base_filename, local_timeout)[source]¶

mycelyso.pilyso.misc.processpool module¶

The processpool submodule contains a custom process-pool, with some additional features not present in standard Python processpool management, e.g. timeouts.

class mycelyso.pilyso.misc.processpool.DuckTypedApplyResult(callable_)[source]¶

Bases: object

fail()[source]¶

get()[source]¶

ready()[source]¶

wait(timeout)[source]¶

class mycelyso.pilyso.misc.processpool.Future[source]¶

Bases: object

args = None¶

command = None¶

dispatch()[source]¶

error = None¶

fail()[source]¶

get()[source]¶

kwargs = None¶

pool = None¶

priority = 0¶

process = None¶

ready()[source]¶

started_at = None¶

status = None¶

timeout = 0¶

value = None¶

wait(time=None)[source]¶

class mycelyso.pilyso.misc.processpool.FutureProcess[source]¶

Bases: multiprocessing.context.Process

RUN = 1¶

STARTUP = 0¶

STOP = 2¶

dispatch()[source]¶

ready()[source]¶

run()[source]¶: Method to be run in sub-process; can be overridden in sub-class

send_command(*args)[source]¶

class mycelyso.pilyso.misc.processpool.InProcessFakePool[source]¶

Bases: object

static advanced_apply(command, args, **kwargs)[source]¶

close()[source]¶

class mycelyso.pilyso.misc.processpool.NormalPool(processes=None, initializer=None, initargs=(), maxtasksperchild=None, context=None)[source]¶

Bases: multiprocessing.pool.Pool

advanced_apply(command, args, **kwargs)[source]¶

class mycelyso.pilyso.misc.processpool.SimpleProcessPool(processes=0, initializer=None, initargs=None, initkwargs=None, future_timeout=0)[source]¶

Bases: object

advanced_apply(command=None, priority=0, args=None, kwargs=None)[source]¶

apply(command, *args, **kwargs)[source]¶

apply_async(fun, args=None, kwargs=None)[source]¶

close()[source]¶

fill_pool()[source]¶

future_became_ready(f)[source]¶

new_process()[source]¶

report_broken_process(p)[source]¶

schedule()[source]¶

exception mycelyso.pilyso.misc.processpool.WrappedException(exception, message='')[source]¶

Bases: RuntimeError

exception = None¶

message = None¶

Module contents¶

The misc submodule contains various functionality, among others the processpool implementation as well as HDF5 output functionality.