The best way to compose music is intuitively. In fact, given what we know about music, and what we don't know about music, it's probably the only way to compose music. (In other words, I make the bold claim that all those people composing the great music that you listen to every day are already doing this.)
Some of the details here are a bit technical, but don't let that put you off trying to understand the basic ideas.
I also give two analogies: that of learning to ride a bike, and exploratory mountain-climbing in dense fog. Neither of these analogies is perfect, but they do help to explain the ideas (and the ways in which the analogies fail are in themselves illuminating).
tl;dr: Summary of Strategies Required to Achieve Intuitive Composition
Without any theory about what music is, the only way you can hope to compose music is by pure trial and error. However, given the nature of the "space" of possible music compositions, composition by trial and error suffers from the problem of combinatorial explosion.
To put it another way, the number of musical items may be very large, but the number of non-musical items is much, much larger. So if you try to compose music by pure trial and error, you will be doing it forever and a day before you even compose one item of music.
We really, really don't know what music is. We might think we know what music is, because we hear it all the time, some of us play it, and we all (well actually most of us) know what it feels like. But that "knowing" is much more subjective than we realise, and from an objective scientific point of view, we really don't know.
That's not to say that one day we won't have a complete scientific theory. But so far no such theory has been discovered, which means that, right now, science can only be of limited help to those of us who wish to compose music.
Existing "music theory", such as you might use when learning to play a music instrument, does have some predictive content, which implies that it does limit the "combinatorial explosion" to some extent. But not enough to make composition by trial and error an efficient process for composing music.
Also, it is entirely possible that there exist novel types of music which traditional music theory actually excludes. To put it another way, traditional music theory is a limited partial theory, which only describes those types of music that we happen to know about already. So by taking existing music theory too seriously, one risks missing the opportunity to compose radically new types of music.
The use of the terms "local" and "neighbourhood" implies that the set of possible musical items exists in some topological space of possible sounds occurring over a particular interval of time, i.e. sound as a function of time (or somewhat simplifying, "sequences of sounds"), and that "musicality" is an approximately continuous function of sound as a function of time.
For those not familiar with the relevant mathematics, this means roughly that the set of possible sounds as a function of time, occurring over some interval of time which is the length that a musical item might be, has a notion of "closeness", where sound functions that are "close" to each other have a similar degree of "musicality".
(Note: the problem of choosing the best notion of "closeness", i.e. the most relevant topological structure, is itself part of the problem of understanding what music is. However, for the purposes of the present discussion, it is probably sufficient that your brain has its own pre-existing notion of "closeness", which is good enough for what is needed to compose music intuitively.)
This is not the only function of the human brain, but it explains our ability to learn various complex behaviours, behaviours which we can learn without necessarily being able to explain to someone else what it is that we have learned. I call this type of learning intuitive learning, because what you learn is an intuition, which you "know", but you can't easily explain to someone else what it is that you know.
One example of intuitive learning is learning how to ride a bike. The set of possible actions taken over some period of time when riding a bike, i.e. the set of all possible muscle movements relevant to steering, pedalling and balancing, is a complex multi-dimensional space, and our success in riding the bike is an approximately continuous function of that behaviour. (To be really precise, we should consider our behaviour when riding a bike to be muscle movement as a function of both time and of all perceptions received relevant to the situation we are in, i.e. from our vision, and from our sense of balance, and from our "kinesthetic sense" of where different parts of our body are. The relevant multi-dimensional space is the set of all such possible functions.)
A "complete" scientific theory of riding a bike would require a complete theoretical knowledge of Newtonian mechanics (and most people who can ride a bike don't know a whole lot about Newtonian mechanics). A complete theory of riding a bike would also include knowledge about all the things you can do with a bicycle, including circus-style tricks, but it is not necessary to know all that just to ride a bicycle from A to B.
(Note: to make this analogy precise, we must compare learning to ride a bicycle to learning to compose one particular tune. The problem of learning how to compose music in general is a problem of learning how to learn, and is more closely analogous to learning how to ride many different types of vehicle, not just bicycles.)
In other words, to learn to ride a bike, you have to actually get on the bike and do stuff, even though the initial stuff you do won't include successfully riding the bike.
In other words, you have to get a "feel" for how music "works", in a particular "region" of musical space.
In practice, this means being willing to "mess around", and play somewhat musically, but at the same time somewhat randomly. Also you should vary your playing randomly in "small" ways, so that you are exploring within one particular "small" region of the musical space, so that the local learning capability of your brain comes into effect.
Intuitive music composition requires exploration of a local musical space, and an awareness of how musical the result is. The only information available to you about how musical a possible musical item is is your own subjective appreciation of that item.
Corollary: to compose, you must use an instrument which "sounds nice". If you attempt to compose on a musical instrument with an inferior sound, important information about the musicality of the composition in progress will be lost, and the brain's intuitive learning functionality will not be able to operate at full efficiency.
The exact scale of "rapid" is uncertain, but of the order of 1 or 2 seconds is a reasonable guess, as this corresponds to the time-scale observed in laboratory learning experiments on non-human mammals. (This is likely relevant, because all mammals have the ability to learn novel physical skills, which presumably involves a mechanism near identical to how humans learn such skills.)
Intuitive music composition requires that you hear the musicality of the music as you are composing it. Which means that you have no choice but to compose by playing.
It is possible to intuitively compose music on a computer, if the software allows you to compose by playing, i.e. you enter data into the software to configure how your playing translates into sound, and then you compose by playing.
But if you type some notes into the software, and then press a "Play" button, the required 1 or 2 seconds has long since passed, and intuitive learning will not occur.
You cannot directly compose music intuitively that other people like, because you don't have any direct access to information about other people's notion of musicality. You can ask them, but the feedback loop will be too slow for intuitive learning to occur.
You can only intuitively compose music that you like. If you manage to compose music that you really like, then almost certainly many other people will also really like it, just because there is a lot of commonality in music taste. (Exactly how much commonality in musical taste exists is difficult to know, because most of the "music" that most of us are familiar with has been composed by someone else, and pre-selected for large-scale popularity, so we have no way of knowing whether or not there exists music which only sounds very musical to a very small number of people, unless we actually compose such music ourselves.)
Intuitive composition requires exploring the neighbourhood of the tune you are trying to compose. But the tune you are trying to compose doesn't yet exist. So you have to explore some neighbourhood, in the hope that some tune can be found within that neighbourhood.
If there is a tune there, the process of exploring will "teach" you which "directions" of change correspond to increases in musicality, and you will be able to head "uphill" to a local "peak" of musicality.
(We can consider this to be an analogy with climbing to the top of a mountain in dense fog, where you don't know how high the mountain is, or even if there is a mountain close enough for you to get there from where you are now. If we revisit the analogy with learning how to ride a bike, an important difference is that even before you start learning to ride a bike, you already know that it is possible to ride a bike, and you already know to some extent what it looks like when other people ride a bike, so the analogy of searching for something that may not exist does not hold so much, unless you happen to be the very first person who ever learned to ride a bike.)
To give a very simplified example: the rhythm of a tune can have its own musicality, somewhat independently of any melodic features, and the melody of a tune may have its own musicality, somewhat independently of any rhythmic features.
It follows that it is useful to explore different musical "neighbourhoods" at the same time, as information about the structure of one neighbourhood will approximately translate to the structure of other neighbourhoods.
Or to put it another way, if you have a tune A and tune B, consider mixing the rhythm of tune A and the melody of tune B, and re-explore the neighbourhood of that mixed tune.
This contradicts the geographical analogy, because it is like climbing a mountain range where different mountains have detailed similarities in structure to each other, which doesn't happen so much in real mountain ranges.
This is an observed feature of music, and it explains, for example, why the use of music in films and television (especially drama) is so ubiquitous.
It follows, given that the successful intuitive composition of music depends on sensitivity to musicality, that the ambitious composer should live a life full of drama with emotional consequences, so that they can exploit this sensitivity to help them in the process of musical composition.
(Note: there is a widespread belief that emotion "inspires" composers and songwriters to "express" their emotions when writing music, which implies that some type of calculation occurs within the brain to translate information about emotions into musical information. However I believe that the concept of increased sensitivity is a more accurate description of what is going on here. In particular this supports the observation that most people do not spontaneously compose new music as a result of feeling such emotions. If there is some kind of "translation", it has to be learned, which cannot be readily distinguished from the process of learning how to compose music.)
Also see my earlier blog article How to Compose Music, which includes some links to musical examples.
A similar concept to "Intuitive" is "Tacit", as in Tacit Knowledge. An interesting paper on that subject is Do we really understand tacit knowledge?.
This manifesto is a Propositional Manifesto. It is licensed under the Creative Commons Attribution-ShareAlike license.