dnikit_tensorflow#

TensorFlow extensions of DNIKit.

class dnikit_tensorflow.TFDatasetExamples[source]#

Bases: object

Example TF Datasets, each bundled as a DNIKit Producer. Loaded from tf.keras.dataset.

class CIFAR10(split_dataset=None, attach_metadata=True, max_samples=-1)#

Bases: _KerasDatasetWithStrLabels

The CIFAR10 dataset, loaded from tf.keras.dataset, that produces Batches.

Metadata labels for this dataset:

['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']

Parameters:

  • split_dataset (Tuple[Tuple[ndarray, ndarray], Tuple[ndarray, ndarray]]) –

    [optional] It is unlikely this parameter will be overridden. This is the dataset as defined by (x_train, y_train), (x_test, y_test), but by default, is set up to load the CIFAR10 dataset from tf.keras.dataset

  • attach_metadata (bool) – [optional] attach metadata <dnikit.base.Batch.metadata to Batch produced by this Producer, under metadata key Batch.StdKeys.LABELS

  • max_samples (int) – [optional] number of samples this Producer should yield (helpful for testing pipelines with a small number of data samples)

class CIFAR100(split_dataset=None, attach_metadata=True, max_samples=-1, label_mode='fine')#

Bases: _KerasDatasetWithStrLabels

Load CIFAR100 dataset, loaded from tf.keras.dataset, that produces Batches.

Metadata labels for this dataset (fine):

['apple', 'aquarium_fish', 'baby', 'bear', 'beaver', 'bed', 'bee', 'beetle', 'bicycle', 'bottle', 'bowl', 'boy', 'bridge', 'bus', 'butterfly', 'camel', 'can', 'castle', 'caterpillar', 'cattle', 'chair', 'chimpanzee', 'clock', 'cloud', 'cockroach', 'couch', 'crab', 'crocodile', 'cup', 'dinosaur', 'dolphin', 'elephant', 'flatfish', 'forest', 'fox', 'girl', 'hamster', 'house', 'kangaroo', 'keyboard', 'lamp', 'lawn_mower', 'leopard', 'lion', 'lizard', 'lobster', 'man', 'maple_tree', 'motorcycle', 'mountain', 'mouse', 'mushroom', 'oak_tree', 'orange', 'orchid', 'otter', 'palm_tree', 'pear', 'pickup_truck', 'pine_tree', 'plain', 'plate', 'poppy', 'porcupine', 'possum', 'rabbit', 'raccoon', 'ray', 'road', 'rocket', 'rose', 'sea', 'seal', 'shark', 'shrew', 'skunk', 'skyscraper', 'snail', 'snake', 'spider', 'squirrel', 'streetcar', 'sunflower', 'sweet_pepper', 'table', 'tank', 'telephone', 'television', 'tiger', 'tractor', 'train', 'trout', 'tulip', 'turtle', 'wardrobe', 'whale', 'willow_tree', 'wolf', 'woman', 'worm']

Metadata labels for this dataset (coarse):

['aquatic_mammals', 'fish', 'flowers', 'food_containers', 'fruit_and_vegetables', 'household_electrical_devices', 'household_furniture', 'insects', 'large_carnivores', 'large_man-made_outdoor_things', 'large_natural_outdoor_scenes', 'large_omnivores_and_herbivores', 'medium_mammals', 'non-insect_invertebrates', 'people', 'reptiles', 'small_mammals', 'trees', 'vehicles_1', 'vehicles_2']

Parameters:
class FashionMNIST(split_dataset=None, attach_metadata=True, max_samples=-1)#

Bases: _KerasDatasetWithStrLabels

Load FashionMNIST dataset, loaded from tf.keras.dataset, that produces Batches.

Metadata labels for this dataset:

['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

Parameters:
class MNIST(split_dataset=None, attach_metadata=True, max_samples=-1)#

Bases: _KerasDatasetLoader

Load MNIST dataset, loaded from tf.keras.dataset, that produces Batches.

Metadata labels for this dataset:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Parameters:
class dnikit_tensorflow.TFModelExamples[source]#

Bases: object

Out-of-the-box TF and Keras models with pre- and post-processing.

MobileNet()#

Load the MobileNet model and processing stages from Keras into DNIKit.

class dnikit_tensorflow.TFModelWrapper(model, preprocessing=None, postprocessing=None)[source]#

Bases: object

A wrapper for loading TensorFlow models into DNIKit Models and PipelineStages, with their pre- and post-processing functions built-in.

Parameters:
__call__(requested_responses=None)[source]#

Generate a PipelineStage that preprocesses Batches for the Model, runs the model with the requested responses, and postprocesses responses before returning them.

Note

If the instance’s postprocessing or preprocessing properties are None, it will ignore those steps`.

Parameters:

requested_responses (None | str | Collection[str]) – passed to the DNIKit Model. Determines which outputs from the model will be present in the Batch output by the resulting PipelineStage.

Returns:

a single PipelineStage or list of PipelineStages

Return type:

PipelineStage | Collection[PipelineStage] | Sequence[PipelineStage | Collection[PipelineStage]]

classmethod from_keras(model, preprocessing)[source]#

Convenience method for loading as TFModelWrapper from Keras models and preprocessors.

Note

When subclassing TFModelWrapper and there are additional pre-postprocessing steps to run outside of Keras’s preprocessing, modify the respective attribute of the return object to add those steps as PipelineStages.

Parameters:

  • model (Model) – TensorFlow Keras model

  • preprocessing (Callable[[ndarray], ndarray]) – keras preprocessing function to transform data

Return type:

TFModelWrapper

static load_keras_model(model)[source]#

Saves TF Keras model to disk and reloads it as a DNIKit Model.

Parameters:

model (Model) – TF Keras model

Return type:

Model

model: Model#

DNIKit Model to put into DNIKit pipeline

postprocessing: None | PipelineStage | Collection[PipelineStage] = None#

One or many DNIKit PipelineStages for post-processing batches after model output

preprocessing: None | PipelineStage | Collection[PipelineStage] = None#

One or many DNIKit PipelineStages for pre-processing batches for this model

property response_infos: Mapping[str, ResponseInfo]#

Get all possible responses in a model. Result is returned as a mapping between response names and the corresponding ResponseInfo.

dnikit_tensorflow.load_tf_model_from_memory(*, session=None, model=None)[source]#

Initialize a TensorFlow Model from a model loaded in memory. This function is supported for both TF2 and TF1, but different parameters are required. For TF2, only pass parameter model. For TF1, only pass parameter session.

Parameters:

  • session (Session | None) – Pass only this parameter when running TensorFlow 1. This is the session that contains the graph to execute.

  • model (Model | None) – Pass only this parameter when running TensorFlow 2. This is the TF Keras model.

Returns:

A TensorFlow Model.

Return type:

Model

dnikit_tensorflow.load_tf_model_from_path(path)[source]#

Initialize a TensorFlow Model from a model serialized in path

Currently accepted serialized model formats, depending on if TF 1 or TF 2 is running.

TF2 Supported formats:

  • TensorFlow Keras SavedModel

  • Keras whole models (h5)

  • Keras models with separate architecture and weights files

TF1 Supported formats:

  • TensorFlow SavedModel

  • TensorFlow checkpoint (pass checkpoint prefix as path param)

  • TensorFlow protobuf

  • Keras whole models

  • Keras models with separate architecture and weights files

Note

The keras loaders are currently using tf.keras instead of keras natively, and so issues might appear when trying to load models saved with native keras (not tf.keras). In this case, load the model outside of DNIKit with keras and pass it to load with load_tf_model_from_memory.

Parameters:

path (str | Path) – Model path (for single model file) or directory that contains all the model files.

Returns:

A DNIKit TensorFlow Model.

Return type:

Model