Imagine looking at an image as a "bag of pixels": shuffle them all up, and look at the resulting image. What do you see? Nothing useful, right?

Now look at a bag-of-words view of a movie review:

    set(['and', 'predecessor,', 'immersive;', 'script,', 'is', 'an', 'engaging', 'as', 'home', 'still', 'its', 'film', 'identity.', 'puts', 'dazzling', 'issues', 'visually', 'colorful', 'While', 'not', 'spin', 'on', 'of', 'while', 'the', 'predictable,'])

It's not certain, but there's definitely a lot more information there.

The predict loop of a linear model works like this (written with sparse vectors, implemented as dictionaries):

    def predict(classes, weights, features):
        scores = {clas: 0 for clas in classes}
        for feature in features:
            for clas, weight in weights[feature].items():
                scores[clas] += weight
        return max(scores, key=lambda clas: scores[clas])

This function is the same for Naive Bayes, Maximum Entropy, linear-kernel SVM, Averaged Perceptron...etc.

All you get to do is attach a weight to each feature, which you'll sum. You can make the features larger or smaller slices of the input, but that's all the structure you get.

Note that linear models have massive disadvantages for speech recognition, too. Linear models don't work very well if you have an analog signal.