New blog post: "Descriptive vs. inferential community detection"

A mini-thread for those too lazy to click the link! 1/7

(Based on recent pre-print: arxiv.org/abs/2112.00183)

skewed.de/tiago/blog/descripti

There are essentially two objectives when doing community detection (or any data analysis): To "describe" or to "infer".

Description involves finding patterns, inference involves finding explanations. 2/7

Most community detection methods out there (modularity, infomap, walktrap, etc.) are descriptive. The communities they find are there, but they cannot explain. In a very concrete sense, they *overfit* your data, confusing actual structure with random fluctuations. 3/7

Follow

Inferential approaches do not do this. They find the most parsimonious explanation for the data — according to Occam's razor — and do not confuse randomness with structure. 4/7

arxiv.org/abs/1705.10225

So, when should we "infer" or "describe"? Here's a good litmus test:

Q: "Would the usefulness of our conclusions change if we learn, after obtaining the communities, that the network being analyzed is completely random?"

5/7

If the answer is "yes", then an inferential approach is needed.

If the answer is "no", then an inferential approach is not required. 6/7

It's arguable that in most scientific contexts the answer would be "no". I this case we need inferential approaches, and descriptive ones are just not up to the task!

There is lots more to say about this. Read it: arxiv.org/abs/2112.00183 7/7

It's arguable that in most scientific contexts the answer would be "yes". I this case we need inferential approaches, and descriptive ones are just not up to the task!

There is lots more to say about this. Read it here: arxiv.org/abs/2112.00183 7/7

Sign in to participate in the conversation
Mastodon @ skewed.de

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!