[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

Zhang-Zhenning · 2022-07-15T09:33:23Z

I have finished exporting savedmodel for models in modelzoo. Under Deeprec/modelzoo there is a Get_SavedModel.md for new user to learn how to export and where the result model will locate.

CLAassistant · 2022-07-15T09:33:29Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

shanshanpt · 2022-07-17T10:09:26Z

for #232

shanshanpt · 2022-07-17T10:10:43Z

modelzoo/BST/pb_to_pbtxt.py

@@ -0,0 +1,13 @@
+from tensorflow.python.saved_model import loader_impl
+from tensorflow.python.lib.io import file_io
+from tensorflow.python.platform import tf_logging as logging


useless file.

shanshanpt · 2022-07-17T10:14:23Z

modelzoo/BST/prepare_savedmodel.py

@@ -0,0 +1,738 @@
+import time
+import argparse
+import tensorflow as tf


不需要修改原文件，在原文件上添加代码即可。

shanshanpt · 2022-07-17T10:16:01Z

modelzoo/BST/start_serving.cc

+#include "serving/processor/serving/processor.h"
+#include "serving/processor/serving/predict.pb.h"
+
+static const char* model_config = "{ \


除fake数据之外，需要根据实际的train data生成serving data，然后能够根据serving data进行process

shanshanpt · 2022-07-17T10:20:24Z

CLA required.

Zhang-Zhenning · 2022-07-25T09:45:27Z

I have finished exporting savedmodel, extracting data from the input file to generate requests and serving for DIEN DeepFM WDL under modelzoo/features/EmbeddingVariable, please check.
One thing that needs to be emphasized for models with non-string inputs like DIEN, is the model_instance.cc will automatically run CreateWarmup to accelerate real serving, which is inappropriate under cases such as the input placeholders don't have a fixed length. During my local test I comment it out, please contact me if there is any problem.

shanshanpt · 2022-07-25T11:30:07Z

modelzoo/features/EmbeddingVariable/DIEN/generate_data.py

+cat_voc = os.path.join(data_location, "cat_voc.pkl")
+
+def prepare_data(input, target, maxlen=None, return_neg=False):
+    # x: a list of sentences


DeepRec中一般使用两个空格锁进，这些文件都稍微改一下

shanshanpt · 2022-07-25T11:33:05Z

modelzoo/features/EmbeddingVariable/DIEN/generate_data.py

+
+
+f.close()
+


多余的空格删除

shanshanpt · 2022-07-25T11:42:17Z

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc

+  ::tensorflow::eas::ArrayShape array_shape;
+  ::tensorflow::eas::ArrayDataType dtype_f =
+            ::tensorflow::eas::ArrayDataType::DT_FLOAT;
+  int num_elem = (int)cur_vector.size();


不用(int)强转，会隐式转换的，或者定义为 size_t num_elem也可以

shanshanpt · 2022-07-25T11:43:15Z

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc

+  int num_elem = (int)cur_vector.size();
+
+  array_shape.add_dim(1);
+  if((int)cur_vector.size() < 0){


if((int)cur_vector.size() < 0) ，cur_vector.size()不会小于0的，类型是size_t本身是正值。

shanshanpt · 2022-07-25T11:44:15Z

modelzoo/features/EmbeddingVariable/DIEN/start_serving_dien.cc

+
+  return input;
+  }
+  array_shape.add_dim((int)cur_vector.size());


cur_vector.size()前面已经赋值了num_elem

shanshanpt · 2022-07-25T11:57:49Z

modelzoo/features/EmbeddingVariable/DeepFM/train.py

@@ -415,6 +415,9 @@ def main(tf_config=None, server=None):

    if tf_config:
        print('train steps : %d' % train_steps)
+        print("-----------")
+        print("-----------")


useless code

shanshanpt · 2022-07-25T11:59:15Z

modelzoo/features/EmbeddingVariable/SavedModel_Serving.md

@@ -0,0 +1,58 @@
+## How to use prepare_savedmodel.py to get savedmodel


文件名叫README.md就可以

shanshanpt · 2022-07-25T12:00:22Z

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc

+
+}
+
+::tensorflow::eas::ArrayProto get_proto_f(float char_input,int dim,::tensorflow::eas::ArrayDataType type){


函数重复

shanshanpt · 2022-07-25T12:00:58Z

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc

+
+          while (record != NULL) {
+              // only 1 label and 39 feature
+              if (j >= 40) break;


shanshanpt · 2022-07-25T12:01:11Z

modelzoo/features/EmbeddingVariable/WDL/start_serving_wdl.cc

+        struct input_format39 inputs;
+        inputs.I1 =  (float)(atof(all_elems[start_idx]));
+        inputs.I2 =  (float)(atof(all_elems[start_idx+1]));
+        inputs.I3 =  (float)(atof(all_elems[start_idx+2]));


这个搞一个循环吧

shanshanpt · 2022-07-28T02:24:31Z

先删除掉不必要的修改。

shanshanpt

先把所有的文件整理一下

shanshanpt · 2022-07-28T12:17:18Z

modelzoo/features/serving/README.md

+## How to use prepare_savedmodel.py to get savedmodel
+
+- Current support model: \
+BST, DBMTL, DeepFM, DIEN, DIN, DLRM, DSSM, ESMM, MMoE, SimpleMultiTask, WDL


Now support DIEN, DeepFM and and WDL.

shanshanpt · 2022-08-08T02:12:57Z

modelzoo/features/serving/DIEN/generate_data.py

+                        maxlen,
+                        data_location=data_location)
+
+f = open("./test_data.csv","w")


test_data.csv 怎么得到的呢，可以在readme中写清楚

shanshanpt · 2022-08-08T02:16:04Z

modelzoo/features/serving/DIEN/prepare_savedmodel.py

+
+    with tf.Session() as sess1:
+
+        model = Model_DIN_V2_Gru_Vec_attGru_Neg(n_uid, n_mid, n_cat,


这个Model_DIN_V2_Gru_Vec_attGru_Neg是哪里import的？

shanshanpt · 2022-08-08T02:17:50Z

modelzoo/features/serving/DIEN/start_serving_dien.cc

+  input.set_dtype(dtype_f);
+
+  switch(dtype_f){
+      case 1:


两个case中大部分代码都在重复的，可以只在
input.add_float_val((float)atof(cur_vector.back()));
input.add_int_val((int)atoi(cur_vector.back()));
代码加上if判断。

shanshanpt · 2022-08-08T02:33:58Z

modelzoo/features/serving/DIEN/start_serving_dien.cc

+          temp_ptrs.clear();
+
+          // traverse current line
+          record = strtok(line, delim);


下面这个可以搞成类似 split(...) 函数吗，现在这样写比较hack。

shanshanpt · 2022-08-08T02:39:09Z

modelzoo/features/serving/DeepFM/start_serving_deepfm.cc

+
+        // input setting
+        ::tensorflow::eas::ArrayProto I1 =  get_proto_cc(&inputs.I1_13[0],1,dtype_f);
+        ::tensorflow::eas::ArrayProto I2 =  get_proto_cc(&inputs.I1_13[1],1,dtype_f);


搞一个数组存储？

shanshanpt · 2022-08-08T02:39:51Z

modelzoo/features/serving/DeepFM/start_serving_deepfm.cc

+int main(int argc, char** argv) {
+
+  // PLEASE EDIT THIS LINE!!!!
+  char filepath[] = "/home/deeprec/DeepRec/modelzoo/features/EmbeddingVariable/WDL/test.csv";


文件如何生成的呢

shanshanpt · 2022-08-08T02:41:04Z

modelzoo/features/serving/README.md

+ For every model listed above, there is a prepare_savedmodel.py. To run this script please firstly ensure you have gotten the checkpoint file from training. To use prepare_savedmodel.py, please use:
+
+  ```
+    cd [modelfolder] 


training的文件位置需要写清楚，用户可能找不到具体的training的文件所在。
或者你把training文件copy到你的目录下也可以。

shanshanpt · 2022-08-08T02:41:51Z

modelzoo/features/serving/WDL/start_serving_wdl.cc

+int main(int argc, char** argv) {
+
+  // PLEASE EDIT THIS LINE!!!!
+  char filepath[] = "/home/deeprec/DeepRec/modelzoo/features/EmbeddingVariable/WDL/test.csv";


文件如何生成的呢

shanshanpt · 2022-08-08T02:42:10Z

modelzoo/features/serving/WDL/start_serving_wdl.cc

+            ::tensorflow::eas::ArrayDataType::DT_INT64;
+
+        // input setting
+        ::tensorflow::eas::ArrayProto I1 =  get_proto_cc(&inputs.I1_9[0],1,dtype_f);


搞一个数组存储

gumanchang · 2023-10-25T09:47:04Z

请问下，这个代码还是没有合并嘛？现在仍然只有训练，没有推理部署，文档也没说

candyzone · 2023-10-25T10:01:26Z

请问下，这个代码还是没有合并嘛？现在仍然只有训练，没有推理部署，文档也没说

DeepRec 本身具备推理的能力，这个PR只是这三个model 的推理例子，你可以参考

shanshanpt self-requested a review July 17, 2022 10:07

shanshanpt reviewed Jul 17, 2022

View reviewed changes

shanshanpt reviewed Jul 25, 2022

View reviewed changes

shanshanpt reviewed Jul 28, 2022

View reviewed changes

reorganize the file structure

576652e

Zhang-Zhenning force-pushed the inference_zzn branch from b52d097 to 576652e Compare July 28, 2022 08:59

shanshanpt reviewed Aug 8, 2022

View reviewed changes

liutongxuan changed the title ~~savedmodel generation~~ [Modelzoo] Add serving for DIEN, DeepFM and WDL. Aug 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

Zhang-Zhenning commented Jul 15, 2022

CLAassistant commented Jul 15, 2022

shanshanpt commented Jul 17, 2022

shanshanpt Jul 17, 2022

shanshanpt Jul 17, 2022

shanshanpt Jul 17, 2022

shanshanpt commented Jul 17, 2022

Zhang-Zhenning commented Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt Jul 25, 2022

shanshanpt commented Jul 28, 2022

shanshanpt left a comment

shanshanpt Jul 28, 2022

shanshanpt Aug 8, 2022

shanshanpt Aug 8, 2022

shanshanpt Aug 8, 2022

shanshanpt Aug 8, 2022

shanshanpt Aug 8, 2022

shanshanpt Aug 8, 2022

shanshanpt Aug 8, 2022

shanshanpt Aug 8, 2022

shanshanpt Aug 8, 2022

gumanchang commented Oct 25, 2023

candyzone commented Oct 25, 2023

		@@ -0,0 +1,58 @@
		## How to use prepare_savedmodel.py to get savedmodel


		}

		::tensorflow::eas::ArrayProto get_proto_f(float char_input,int dim,::tensorflow::eas::ArrayDataType type){


		with tf.Session() as sess1:

		model = Model_DIN_V2_Gru_Vec_attGru_Neg(n_uid, n_mid, n_cat,



		f.close()

[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

Are you sure you want to change the base?

[Modelzoo] Add serving for DIEN, DeepFM and WDL. #319

Conversation

Zhang-Zhenning commented Jul 15, 2022

CLAassistant commented Jul 15, 2022

shanshanpt commented Jul 17, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shanshanpt commented Jul 17, 2022

Zhang-Zhenning commented Jul 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shanshanpt commented Jul 28, 2022

shanshanpt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gumanchang commented Oct 25, 2023

candyzone commented Oct 25, 2023