It doesn't make me mad, but it mildly irks me when people comment on my videos with just a time. Why did you put "3:47" as your comment? Was it funny? Was it not funny? Was that when you gave up watching?
I mean I think it's because they're pointing out a funny part, but I just have no idea.
I'm sorry OP, but I'm not sure you under stand what is the purpose of comments - people reacting to the video, meaning people watch the video and then say what's on their mind regarding the video (I feel silly explaining this lol) if you don't want to see spoilers on the video watch it first.
Some number of these are bots making fake accounts look like legitimate users. Eventually they'll be making pro-China/Russia/Israel/Whatever comments on political videos.