> A signal informs to the degree it resists prediction.
Let me offer a counter-point: A signal informs to the degree it lets the recipient infer the intention of the sender
A signal has mathematical properties. For instance the vatiation of some attribute of it can be completely "random" which I guess means a uniform flat distribution of attribute values.
But if we can infer the properties of that distribution we already have a prediction for the signal: It's probabaility distribution is and as far as we can predict (!) is and stays the same, or so we predict.
So a complete randomness does not make the a signal resistant to prediction, in fact it lets us predict that the signal is and stays completely random.
So it is not about "unpredictability" per se but about unpredictability of some mathematical property of the signal, its probability distribution for instance. So to make a signal carry maximum information we would need to produce a signal whose propability distribution varies unpredictably!
But the ppoperties whose unpredictability we measure can be chosen arbitrarily and depending on their choice the amoount of information we measure for a signal (according to Shannon) depends on those choices. So there is no single unambiguous way to measuere the amount of information in a signal -- without considering the actual intention of whoever created the signal.
Shannon's insights about information are great but they are largely insiights about mathematics, disjunct from "information" in the real world, as we understand it.
> A signal informs to the degree it resists prediction.
Let me offer a counter-point: A signal informs to the degree it lets the recipient infer the intention of the sender
A signal has mathematical properties. For instance the vatiation of some attribute of it can be completely "random" which I guess means a uniform flat distribution of attribute values.
But if we can infer the properties of that distribution we already have a prediction for the signal: It's probabaility distribution is and as far as we can predict (!) is and stays the same, or so we predict.
So a complete randomness does not make the a signal resistant to prediction, in fact it lets us predict that the signal is and stays completely random.
So it is not about "unpredictability" per se but about unpredictability of some mathematical property of the signal, its probability distribution for instance. So to make a signal carry maximum information we would need to produce a signal whose propability distribution varies unpredictably!
But the ppoperties whose unpredictability we measure can be chosen arbitrarily and depending on their choice the amoount of information we measure for a signal (according to Shannon) depends on those choices. So there is no single unambiguous way to measuere the amount of information in a signal -- without considering the actual intention of whoever created the signal.
Shannon's insights about information are great but they are largely insiights about mathematics, disjunct from "information" in the real world, as we understand it.
Or, am I "misinformed"? :-)