machine-learning-for-networking.md (1755B)
1 +++ 2 title = 'Machine learning for networking' 3 +++ 4 5 ## Machine learning for networking 6 ### Video streaming 7 Dynamic streaming over HTTP (DASH). 8 9 Reinforcement learning: reward measures how good an action is, goal is to maximize cumulative reward. 10 11 Pensieve: learning-based ABR algorithm 12 - ABR: adaptive bit rate 13 - Pensieve learns it through experience 14 - RL reward is `bitrate - rebuffering - smoothness` 15 - use large corpus of network traces to train the model 16 17 ### Network packet classification 18 Decides action to take based on matched rule. 19 Solutions: 20 - hardware-based (e.g. TCAM): fast. but expensive, energy-consuming, hard to scale 21 - software-based (e.g. decision-tree): scalable, but slow and requires large memory 22 23 Existing techniques: 24 - node cutting: cut space into smaller areas, each area corresponds to leaf in decision tree, match by traveling through decision tree and select rule in matched leaf with highest priority 25 - rule partitioning: partition space into two parts, build separate decision (sub)tree for each part, match by traveling through decision tree and select rule in matched leaf with highest priority 26 27 End-to-end learning: replace decision tree with RL model 28 - may not need data structure at all 29 - but cannot guarantee classification correctness, packet inference takes too long, needs specialised hardware 30 31 NeuroCuts: use deep reinforcement learning to tackle problem of building decision trees 32 - action: either cutting node, or partitioning set of rule 33 - reward: classification time, or memory footprint, or both 34 - reward delayed and only given when whole tree is built 35 36 Naive MDP formulation: 37 - sequential markov decision process 38 - assumes DFS order of building the tree node-by-node 39 - action: cut or partition current node