Resistor sizes are primarliy governed by their power dissipation in the circuit. Secondary criterion is voltage rating i.e. the spacing between the pads. Capacitor sizes are primarily governed by their capacitance and voltage rating but that varies tremendously with the type of capacitor (aluminium or tantalum electrolytic, ceramic disc or multilayer, Mylar, Metalized Polypropylene, etc. And the choice of type of capacitor depends very much on the application. In switch mode supplies, the ripple current rating and especially for electrolytics, lifetime is equally important with voltage rating. Longer lifetime electrolytic capacitors tend to be physically bigger. Inductor sizes are highly application dependent. Inductance and current ratings primarliy govern size but physical properties have a huge influence on them. Electronics is not like building with Lego: component choices and understanding what drives those choices are a major part of electronic design. It pays to read device application notes carefully as they often give guidance on component choices for the surrounding circuitry such as decoupling capacitors and input and output capacitors for regulators, inductors for switchers and so on. Having said all that, at the end of the design process, the availability of parts may be the final arbiter of the size and type of parts because you can only choose from what is available. That can be almost as time consuming as doing the design. :)

Good points from Andy. Definitely need to know how each characteristic of a component impacts the circuit. But after you know and understand that, there is also the manufacturability of your device. Who is going to assemble it? Who will solder all the parts? What are the parts? THT or SMD. If they are SMD, how small are they? Will you be able to solder them yourself or even if you get them soldered for you, will you be able to rework them if necessary? There are rulers that have most component footprints on them so you can see how big they are in life size. I'd suggest you buy at least one like this:  Also, if you are making the PCB at home, depending on you skill and manufacturing process, I'd suggest using 0805 passive SMD parts as default, SO-xx packages, TQFP-32 are really easy to produce and solder. Your part size also depends on your PCB size. does it have to be as small as possible? Are there size constraints or does the PCB have to be a certain shape or can it be as large as you please? There is also no need to use the largest components or smallest components for the sake of having them. If its a home project, I usually go for 0805 passives because I bought a few passives "books" with capacitors and resistors in many values. If it's work related, you usually need to meet a lot of standards and size constraints. Now I'll need some 0402 size passives because we are making a wearable and you don't want anything bulky on your clothes. I'd also suggest buying some soldering kits to see what size components you can solder with bare eyes. Hope that also gave you some insight :D