In responding to your question, I'm hoping to find some resolution on this for myself, too, just so you know.
These people you're needing to small-talk with...are they complete strangers, or people you already know?
I think Angela and Harrison nailed it when it comes to strangers where there's really no opportunity for the relationship to ever go any further than small talk, then you never see them again. Develop a script that you can follow like an "if-then" flow chart or something, and use it as often as you need it. So long as the person hasn't heard it before, you'll eventually fine-tune it enough so it works for you. Some of it can be used repeatedly with the same person if there's enough time-lapse between the conversations (weather, well-being, and work all tend to change often enough, you can ask for an "update", lol).
One problem with this, you've mentioned already: It feels phony. But really, that's all you can expect from most relationships. Most people can't stand the thought of being vulnerable with strangers, and that's a healthy sense of boundaries. Depending on how far out a person is from your core vulnerability/intimacy layer, determines how much access that person has to the "real" you. Think about this "real" you being at the center of several concentric circles, and each one represents a different layer of intimacy/vulnerability in the relationship. Most people stay in the outer layers. So if the conversation never gets past the weather, that's okay. They haven't earned the right to have access to more intimate knowledge of what makes you tick. And the weather is reality (not phony)...just not particularly important (unless it's your obsession of the moment, lol).
Another problem I've found is that it's not only complete strangers in one-time meetings where small-talk is required. Weekly church meetings, family reunions, business conferences (in my field, we see a lot of the same people at these conferences every year), and even a family birthday party all throw demands for small talk in my face with people that have already heard my script, sometimes several times over.
There are some people that I've decided...they're not "safe" enough to give them access to my inner layers of vulnerability. Some of these people know me very well (my parents and sisters), but they can't be trusted anymore to treat intimate knowledge of my thoughts with respect. So I don't go there with them anymore. Small talk with them is very painful, because they keep trying to dig, and I keep having to keep them out. I want to be honest, but I also want to be safe and enforce the boundaries I've decided I need to have with them.
There are other people...friends at church, for example...who I'd like to give them access to my more vulnerable layers. But I've learned you can't just throw everything out there on the table or people feel intimidated. There has to be a more leisurely pace to this disclosure or people feel threatened and move away. So it's a delicate balance.
In each conversation with these people, it's like we start at a slightly more outer layer at first, weave through some of that, then there's an opening to go a little deeper with honesty and vulnerability. I look for the open door, or open a couple of doors of my own, and see if they respond. It's kind of like a dance. The "small talk" helps keep the conversation balanced and safe and grounded, so we can explore deeper layers of vulnerability. Then we back off again before finishing the conversation. And sometimes, the timing isn't right for some reason, so the open doors get ignored. And that's okay, because the surrounding environment's effect on our sense of safety in the conversation
is "real"...not necessarily conducive to vulnerability, but no less a part of reality (and therefore not "phony").
Honestly, even these conversations feel "phony" in many ways. But eventually, at some point, we reach a place of comfort and vulnerability that conversations start to get more real and vulnerable. It just takes time.
And then there are people who I know (so they've heard my script) but we don't know each other very well, and the nature of the relationship dictates that we can never have a very deep relationship (like business contacts at conferences). I don't have any ingenious solutions...I keep a few key questions in my head, look for more questions to ask as the conversation progresses, and try to get them talking. If I can find something they're passionate about--even if it's not the business topic at hand--I'll work at getting them talking about it then just listen. That's a lot safer than sticking my foot in my mouth with the stupid things that tend to slip out when I try to talk without a script or focused purpose. The challenge, then, is to remember what they told me and who it was so I'll have that data handy the next time I talk with them and can build on it, or at least not overlap. Not easy!