When it comes to the problem of high disk io on yarn cluster, it may lead to many undesirable situations, such as application’s slowing down too much, hdfs operation shows high latency, although the file being operated is very small. What happened? It may be a shuffle problem. For example, spark is our main compute […]