Actually, you're exactly right. Think of it this way. Liberal/Conservative is a left/right issue. Now, what is the opposite of a libertarian? A statist. Someone that wants complete domination of individual life by government (of course, there's much middle ground between the two). Now that isn't left right, rather, it's up and down. Say, libertarian is up higher, for more personal freedom, and statist is down lower, for less personal freedoms. Thus, what you end up having, are conservative libertarians, liberal libertarians, conservative statists, and liberal statists. Every one of them will have a very different view of things. then, there's all the middle grounds which they encroach upon.
So, it may look something like this:
Of course, to libertarians, it looks more like this:
So, for instance, a typical "conservative" may be very much against Gay marriage, but he doesn't care what happens in the bedroom. A libertarian conservative may be against gay marriage, but would vote for it because it puts the right at the individual level and he feels that it is wrong to deny that right to another person. A statist conservative may not only be against gay marriage, but believe the government has the right to dictate that homosexuality be punished.
A liberal libertarian may be for both gay marriage and believe that homosexuality is absolutely fine, but will fight tooth and nail for the conservatives to be able to speak their mind about it, even though they completely disagree. A regular liberal is all for it and looks on with disgust at the conservatives. A statist liberal not only is for it, but believes the conservatives need to be shut up by government censorship.
That is just one scenario.
Here is a "Libertarian Purity Test." It's fun and enlightening at the same time. I'm a "Medium-core libertarian." According to this test.