Diameter is important, but if you're trying to get any magnification out of that system, you need to look at the focal length.
If you have a very distant light source, like the Sun, focused through a convex lens, until the image of the source is as small and as clear as possible, then the focal length is the distance between the lens and the image you're creating. Like when you burn paper with the lens, focal length is the lens-paper distance.
For concave lenses focal length is a little more tricky to define, but they do have a focal length anyway.
If F is the focal length of the forward lens (the objective), and f is the focal length of the lens near your eye (the ocular, or the eyepiece), then the magnification of the instrument is:
M = F / f
So you need an objective with a long focal length, and an eyepiece with a short focal length.
Both the objective and the eyepiece could be convex actually. It's just that the image will be reversed. If the eyepiece is concave, the image will be straight-up, but the field of view of the telescope will be narrow (like looking through a peephole).
If you use both convex lenses, the distance between them need to be close to the sum of their focal lengths, F + f. Start there and adjust it slightly for best results.
If you use a concave lens for the eyepiece, then the distance between lenses needs to be the difference of their focal lengths, F - f.
Both lenses need to be perpendicular on their common axis, to minimize aberrations.
It helps if they are mounted in a tube, to reduce glare, but it's not absolutely necessary. They could simply be attached to a long rod or something - but you'll get a lot of glare into the eyepiece that way.
Do not necessarily try to get very high magnification. If the diameter of the objective is D, measured in mm, then the maximum usable magnification is 2D. So an objective 100 mm in diameter could never provide more than 200x magnification - I mean it could, but the image would be blurry. Even then, this is a theoretical limit, and in practice the image will get pretty bad long before you reach the limit, unless you use high quality lenses designed specifically for telescopes, etc. Also, at high magnification it's difficult to hold the scope steady, and the image is jumping all over the place.
Try and make a 5x ... 8x instrument first, see how that goes. A lot of binoculars are in this range.
I've built several refractor telescopes from lenses when I was in high school. It's totally doable. It just takes some patience to figure out how to solve all problems.