[myawesometmlsolution-3f10] Details ============================ Generated On: 2026-06-05 14:00:52 UTC TML Solution DAG Parameters' Details: User Chosen Parametets ---------------------------- STEP 1: Get TML Core Params: `tml_system_step_1_getparams_dag `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - solutionname - myawesometmlsolution-3f10 * - solutiontitle - My Solution Title * - solutiondescription - This is an awesome real-time solution built by TSS * - brokerhost - 127.0.0.1 * - brokerport - 9092 * - cloudusername - None * - ingestdatamethod - LOCALFILE STEP 2: Create Kafka Topics: `tml_system_step_2_kafka_createtopic_dag `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - companyname - Otics * - myname - Sebastian * - myemail - Sebastian.Maurice * - mylocation - Toronto * - replication - 1 * - numpartitions - 1 * - enabletls - 1 * - microserviceid - * - raw_data_topic - iot-raw-data * - preprocess_data_topic - iot-preprocess,iot-preprocess2 * - ml_data_topic - ml-data * - prediction_data_topic - prediction-data STEP 3: `Produce to Kafka Topics `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - PRODUCETYPE - LOCALFILE * - inputfile - /rawdatademo/IoTData.txt * - TOPIC - iot-raw-data * - PORT - _5050 * - IDENTIFIER - TML solution,/rawdatademo/IoTData.txt * - HTTPADDR - https:// * - FROMHOST - ('eb95da050ee4', '172.17.0.2') * - TOHOST - 0.0.0.0 * - CLIENTPORT - Not Applicable * - TSS_CLIENTPORT - Not Applicable * - TML_CLIENTPORT - Not Applicable * - docfolder - * - doctopic - * - chunks - 0 * - docingestinterval - 0 STEP 4: Preprocesing Data: `tml-system-step-4-kafka-preprocess-dag `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - raw_data_topic - iot-raw-data * - preprocess_data_topic - iot-preprocess,iot-preprocess2 * - preprocessconditions - * - delay - 70 * - maxrows - 800 * - array - 0 * - saveasarray - 1 * - topicid - -999 * - rawdataoutput - 1 * - asynctimeout - 120 * - timedelay - 0 * - preprocesstypes - anomprob,trend,avg * - pathtotmlattrs - --pathtotmlattrs-- * - identifier - IoT device performance and failures * - jsoncriteria - uid=metadata.dsn,filter:allrecords~subtopics=metadata.property_name~values=datapoint.value~identifiers=metadata.display_name~datetime=datapoint.updated_at~msgid=datapoint.id~latlong=lat:long STEP 4a: Preprocesing Data: `tml-system-step-4a-kafka-preprocess-dag `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - raw_data_topic - --raw_data_topic1-- * - preprocess_data_topic - --preprocess_data_topic1-- * - preprocessconditions - --preprocessconditions1-- * - delay - --delay1-- * - maxrows - --maxrows1-- * - array - --array1-- * - saveasarray - --saveasarray1-- * - topicid - --topicid1-- * - rawdataoutput - --rawdataoutput1-- * - asynctimeout - --asynctimeout1-- * - timedelay - --timedelay1-- * - preprocesstypes - --preprocesstypes1-- * - pathtotmlattrs - --pathtotmlattrs1-- * - identifier - --identifier1-- * - jsoncriteria - --jsoncriteria1-- STEP 4b: Preprocesing Data: `tml-system-step-4b-kafka-preprocess-dag `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - raw_data_topic - --raw_data_topic2-- * - preprocess_data_topic - --preprocess_data_topic2-- * - preprocessconditions - --preprocessconditions2-- * - delay - --delay2-- * - maxrows - --maxrows2-- * - array - --array2-- * - saveasarray - --saveasarray2-- * - topicid - --topicid2-- * - rawdataoutput - --rawdataoutput2-- * - asynctimeout - --asynctimeout2-- * - timedelay - --timedelay2-- * - preprocesstypes - --preprocesstypes2-- * - pathtotmlattrs - --pathtotmlattrs2-- * - identifier - --identifier2-- * - jsoncriteria - --jsoncriteria2-- STEP 4c: Preprocesing Data: `tml-system-step-4c-kafka-preprocess-dag `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - raw_data_topic - --raw_data_topic3-- * - preprocess_data_topic - --preprocess_data_topic3-- * - delay - --delay3-- * - maxrows - --maxrows3-- * - array - --array3-- * - saveasarray - --saveasarray3-- * - topicid - --topicid3-- * - rawdataoutput - --rawdataoutput3-- * - asynctimeout - --asynctimeout3-- * - timedelay - --timedelay3-- * - searchterms - --rtmssearchterms-- * - rtmsstream - --rtmsstream-- * - identifier - --identifier3-- * - rememberpastwindows - --rememberpastwindows-- * - patternwindowthreshold - --patternwindowthreshold-- * - localsearchtermfolder - --localsearchtermfolder-- * - localsearchtermfolderinterval - --localsearchtermfolderinterval-- * - rtmsscorethreshold - --rtmsscorethreshold-- * - rtmsscorethresholdtopic - --rtmsscorethresholdtopic-- * - attackscorethreshold - --attackscorethreshold-- * - attackscorethresholdtopic - --attackscorethresholdtopic-- * - patternscorethreshold - --patternscorethreshold-- * - patternscorethresholdtopic - --patternscorethresholdtopic-- * - rtmsfoldername - --rtmsfoldername-- * - rtmsmaxwindows - --rtmsmaxwindows-- * - RTMS Output Github Link - `Output Data URL <--rtmsoutputurl-->`_ STEP 5: Entity Based Machine Learning : `tml-system-step-5-kafka-machine-learning-dag `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - preprocess_data_topic - iot-preprocess,iot-preprocess2 * - ml_data_topic - ml-data * - modelruns - --modelruns-- * - offset - -1 * - islogistic - --islogistic-- * - networktimeout - --networktimeout-- * - modelsearchtuner - --modelsearchtuner-- * - processlogic - --processlogic-- * - dependentvariable - --dependentvariable-- * - independentvariables - --independentvariables-- * - rollbackoffsets - --rollbackoffsets-- * - topicid - -999 * - consumefrom - --consumefrom-- * - fullpathtotrainingdata - --fullpathtotrainingdata-- * - transformtype - --transformtype-- * - sendcoefto - --sendcoefto-- * - coeftoprocess - --coeftoprocess-- * - coefsubtopicnames - --coefsubtopicnames-- * - ML Output Github Link - `Output Data URL <--mloutputurl-->`_ STEP 6: Entity Based Predictions: `tml-system-step-6-kafka-predictions-dag `_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - preprocess_data_topic - iot-preprocess,iot-preprocess2 * - ml_prediction_topic - --ml_prediction_topic-- * - streamstojoin - --streamstojoin-- * - inputdata - --inputdata-- * - consumefrom - --consumefrom2-- * - offset - -1 * - delay - 70 * - usedeploy - --usedeploy-- * - networktimeout - --networktimeout-- * - maxrows - 800 * - topicid - -999 * - pathtoalgos - --pathtoalgos-- STEP 7: Real-Time Visualization: `tml-system-step-7-kafka-visualization-dag `_ ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - vipervizport - 9689 * - topic - iot-preprocess * - dashboardhtml - dashboard.html * - secure - 1 * - offset - -1 * - append - 0 * - chip - amd64 * - rollbackoffset - 400 STEP 8: `tml_system_step_8_deploy_solution_to_docker_dag `_ ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - Docker Container - --dockercontainer-- * - Docker Run Command - --dockerrun-- STEP 9: `tml_system_step_9_privategpt_qdrant_dag `_ ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - PrivateGPT Container - --pgptcontainername-- * - PrivateGPT Run Command - --privategptrun-- * - Qdrant Container - --qdrantcontainer-- * - Qdrant Run Command - --qdrantrun-- * - Consumefrom - --consumefrom-- * - pgpt_data_topic - --pgpt_data_topic-- * - offset - -1 * - rollbackoffset - 400 * - topicid - -999 * - enabletls - 1 * - partition - --partition-- * - prompt - --prompt-- * - context - --context-- * - jsonkeytogather - --jsonkeytogather-- * - keyattribute - --keyattribute-- * - keyprocesstype - --keyprocesstype-- * - vectordbcollectionname - --vectordbcollectionname-- * - concurrency - --concurrency-- * - CUDA_VISIBLE_DEVICES - --cuda-- * - pgpthost - --pgpthost-- * - pgptport - --pgptport-- * - hyperbatch - --hyperbatch-- * - docfolder - --docfolder-- * - docfolderingestinterval - --docfolderingestinterval-- * - useidentifierinprompt - --useidentifierinprompt-- * - searchterms - --searchterms-- * - streamall - --streamall-- * - temperature - --temperature-- * - vectorsearchtype - --vectorsearchtype-- * - llm - --llmmodel-- * - embedding - --embedding-- * - vectorsize - --vectorsize-- * - contextwindowsize - --contextwindowsize-- * - vectordimension - --vectordimension-- * - mitrejson - --mitrejson-- STEP 9b: `tml_system_step_9b_agenticai_dag `_ ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - rollbackoffset - --agenticai-rollbackoffset-- * - ollama-model - --agenticai-ollama-model-- * - deletevectordbcount - --agenticai-deletevectordbcount-- * - vectordbpath - --agenticai-vectordbpath-- * - temperature - --agenticai-temperature-- * - topicid - --agenticai-topicid-- * - enabletls - --agenticai-enabletls-- * - partition - --agenticai-partition-- * - vectordbcollectionname - --agenticai-vectordbcollectionname-- * - ollamacontainername - --agenticai-ollamacontainername-- * - mainip - --agenticai-mainip-- * - mainport - --agenticai-mainport-- * - embedding - --agenticai-embedding-- * - agenttopic - --agenticai-agenttopic-- * - agents_topic_prompt - --agenticai-agents_topic_prompt-- * - teamlead_topic - --agenticai-teamlead_topic-- * - teamleadprompt - --agenticai-teamleadprompt-- * - supervisor_topic - --agenticai-supervisor_topic-- * - supervisorprompt - --agenticai-supervisorprompt-- * - agenttoolfunctions - --agenticai-agenttoolfunctions-- * - agent_team_supervisor_topic - --agenticai-agent_team_supervisor_topic-- * - concurrency - --agenticai-concurrency-- * - CUDA_VISIBLE_DEVICES - --agenticai-cuda-- * - contextwindow - --agenticai-contextwindow-- * - localmodelsfolder - --agenticai-localmodelsfolder-- STEP 10: `tml_system_step_10_documentation_dag `_ ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - **User Parameter** - **Chosen Value** * - Solution Documentation URL - https://myawesometmlsolution-3f10.readthedocs.io