cluster.neuralnet package¶
Submodules¶
cluster.neuralnet.neuralnet_node module¶
-
class
cluster.neuralnet.neuralnet_node.
NeuralNetNode
[source]¶ Bases:
cluster.common.common_node.WorkFlowCommonNode
-
check_batch_exist
(node_id)[source]¶ use if you want to check batch data exists or not check if batch version data exists :param node_id: :return:
-
cluster.neuralnet.neuralnet_node_attnseq2seq module¶
-
class
cluster.neuralnet.neuralnet_node_attnseq2seq.
NeuralNetNodeAttnSeq2Seq
[source]¶ Bases:
cluster.neuralnet.neuralnet_node.NeuralNetNode
-
eval
(node_id, conf, data=None, result=None)[source]¶ eval result wit test data :param node_id: :param parm: :return:
-
cluster.neuralnet.neuralnet_node_autoencoder module¶
-
class
cluster.neuralnet.neuralnet_node_autoencoder.
NeuralNetNodeAutoEncoder
[source]¶ Bases:
cluster.neuralnet.neuralnet_node.NeuralNetNode
this is a network class for Autoencoder Autoencoder provide two types of service 1. provide matrix size of input matrix 2. provide compressed matrix
-
anomaly_detection
(node_id, parm={'input_data': {}, 'type': 'encoder'}, raw_flag=False)[source]¶ this is a function that judge requested data is out lier of not :param node_id: string :param parm: dict (include input data) :return: boolean
-
eval
(node_id, conf_data, data=None, result=None, stand=0.1)[source]¶ eval process check if model works well (accuracy with cross table) :param node_id: :param conf_data: :param data: :param result: :return:
-
cluster.neuralnet.neuralnet_node_bilstmcrf module¶
-
class
cluster.neuralnet.neuralnet_node_bilstmcrf.
NeuralNetNodeBiLstmCrf
[source]¶ Bases:
cluster.neuralnet.neuralnet_node.NeuralNetNode
,cluster.common.neural_common_bilismcrf.BiLstmCommon
-
eval
(node_id, conf_data, data=None, result=None, stand=0.1)[source]¶ eval process check if model works well (accuracy with cross table) :param node_id: :param conf_data: :param data: :param result: :return:
-
get_feed_dict
(words, labels=None, lr=None, dropout=None)[source]¶ Given some data, pad it and build a feed dictionary Args:
- words: list of sentences. A sentence is a list of ids of a list of words.
- A word is a list of ids
labels: list of ids lr: (float) learning rate dropout: (float) keep prob
- Returns:
- dict {placeholder: value}
-
predict
(node_id, parm={'input_data': {}})[source]¶ predict logic for ner tockenize input text and find matching tags for each value :param node_id: :param parm: :return:
-
predict_batch
(sess, words)[source]¶ - Args:
- sess: a tensorflow session words: list of sentences
- Returns:
- labels_pred: list of labels for each sentence sequence_length
-
run_epoch
(sess, train, dev, tags, epoch)[source]¶ Performs one complete pass over the train set and evaluate on dev Args:
sess: tensorflow session train: dataset that yields tuple of sentences, tags dev: dataset tags: {tag: index} dictionary epoch: (int) number of the epoch
-
cluster.neuralnet.neuralnet_node_cnn module¶
cluster.neuralnet.neuralnet_node_d2v module¶
-
class
cluster.neuralnet.neuralnet_node_d2v.
NeuralNetNodeDoc2Vec
[source]¶
cluster.neuralnet.neuralnet_node_fasttext module¶
cluster.neuralnet.neuralnet_node_kerasdnn module¶
-
class
cluster.neuralnet.neuralnet_node_kerasdnn.
NeuralNetNodeKerasdnn
[source]¶
cluster.neuralnet.neuralnet_node_residual module¶
cluster.neuralnet.neuralnet_node_rnn module¶
cluster.neuralnet.neuralnet_node_seq2seq module¶
-
class
cluster.neuralnet.neuralnet_node_seq2seq.
NeuralNetNodeSeq2Seq
[source]¶
cluster.neuralnet.neuralnet_node_w2v module¶
-
class
cluster.neuralnet.neuralnet_node_w2v.
NeuralNetNodeWord2Vec
[source]¶
cluster.neuralnet.neuralnet_node_wcnn module¶
-
class
cluster.neuralnet.neuralnet_node_wcnn.
NeuralNetNodeWideCnn
[source]¶ Bases:
cluster.neuralnet.neuralnet_node.NeuralNetNode
-
eval
(node_id, conf_data, data=None, result=None)[source]¶ eval process check if model works well (accuracy with cross table) :param node_id: :param conf_data: :param data: :param result: :return:
-
cluster.neuralnet.neuralnet_node_wdnn module¶
-
class
cluster.neuralnet.neuralnet_node_wdnn.
NeuralNetNodeWdnn
[source]¶ Bases:
cluster.neuralnet.neuralnet_node.NeuralNetNode
-
generator_len
(it)[source]¶ Help for Generator length promote util class(?) :param it : python generator :return: length of generatorDataNodeFrame
-
cluster.neuralnet.resnet module¶
-
class
cluster.neuralnet.resnet.
ResnetBuilder
[source]¶ Bases:
object
-
static
build
(input_shape, num_outputs, block_fn, repetitions)[source]¶ Builds a custom ResNet like architecture.
- Args:
input_shape: The input shape in the form (nb_channels, nb_rows, nb_cols) num_outputs: The number of outputs at final softmax layer block_fn: The block function to use. This is either basic_block or bottleneck.
The original paper used basic_block for layers < 50- repetitions: Number of repetitions of various block units.
- At each block unit, the number of filters are doubled and the input size is halved
- Returns:
- The keras Model.
-
static
-
cluster.neuralnet.resnet.
basic_block
(filters, init_strides=(1, 1), is_first_block_of_first_layer=False)[source]¶ Basic 3 X 3 convolution blocks for use on resnets with layers <= 34. Follows improved proposed scheme in http://arxiv.org/pdf/1603.05027v2.pdf
-
cluster.neuralnet.resnet.
bottleneck
(filters, init_strides=(1, 1), is_first_block_of_first_layer=False)[source]¶ Bottleneck architecture for > 34 layer resnet. Follows improved proposed scheme in http://arxiv.org/pdf/1603.05027v2.pdf
- Returns:
- A final conv layer of filters * 4