I'm in the middle of an apartment search. I was reflecting the other day on how much searching for an apartment has changed over the last 10 years.
Conclusion: it hasn't changed much at all.
It's still necessary to search on a myriad of sites. Most of the time, by the time you inquire, a place is gone. Or the ad was misleading and that "$2000 2BR" is actually "$2000 for the cheapest Studio we have, but we also have 2BR for $3.5K".
Finding places that match my criteria still requires careful inspection of the description. Checking a filter box for "Parking" excludes places that only mention parking in the description.
With all of the modernization and streamlining that has happened in other parts of our lives, apartment hunting remains unstructured, messy, frustrating, stressful.
Aggregators like Padmapper helped for awhile, but these days, it seems like most listings are posted by scammy middlemen just trying to make a commission on the listing, and now the site actively makes the process even harder.
Is hunting for an apartment doomed to be this messy forever? It seems like an area that is ripe for disruption or a better option, but what would it take to actually solve this?
---
With all of that said, this post is exactly what I needed right now. I've been toying with a very similar idea to improve the efficiency of my search process and this is good inspiration.
I used to work at a startup that aggregated apartment listings. Long gone now, we couldn't compete with Zillow or Apartments.com. But what we were doing is just aggregating all of the rental websites we could possibly scrape into one interface.
It was hard. There are so many scams out there. If you are not hand curating listings, you have to rely on somewhat novel approaches to filter out all the bad data. For example, any mention of 'Jesus' or 'God' automatically blocked the listing. Sure there might have been legitamate listings, but anytime I would go in and check 99% of the time it was a scam. You also have scams of people listing units they don't own or have any relation too. They ask for a security deposit up front and just pocket it, leaving the new tenant to have an awkward conversation with the real resident.
Data is often unformatted too. Scraping out bedrooms and bathrooms can even be a challenge on websites like craigslist where the listing is just one big paragraph. (Not anymore, craigslist has come a long way, but that's how it was 10 years ago). Often times you had to just search for the closest number around the word "bed" or "bath". Don't even think about getting features like "driveway" or "laundry" out of them.
In the end, we ended up utilizing some pretty intense ETL pipelines to collate historical data, census information, property assessment data, and other things to try to get a more accurate picture on our listings.
But that didn't win out. What won out are the sites like Apartments.com or Zillow, where legitimate property owners can post their listings in a formatted searchable way. We could scrape them and post the same listings on our site, but at that point we were just pushing our customers to another platform that honestly worked better than our own.
We couldn't have the most up to date data, that was determined by how fast we could go back to scrape a listing. And often times we were knee deep in a battle to avoid being blocked by these companies. Often times, after we had exhausted our proxies, the only thing left to use was Tor.
> I used to work at a startup that aggregated apartment listings. Long gone now, we couldn't compete with Zillow or Apartments.com. But what we were doing is just aggregating all of the rental websites we could possibly scrape into one interface.
[...]
> We couldn't have the most up to date data, that was determined by how fast we could go back to scrape a listing. And often times we were knee deep in a battle to avoid being blocked by these companies. Often times, after we had exhausted our proxies, the only thing left to use was Tor.
The flip side is if you are looking at listing from small property management companies, they are probably low on resources. The websites aren't very well optimized to serve thousands of concurrent users. Every request to search inventory hits SQL Server which is probably one small box with no automatic failover. I don't like it but unless we somehow help these people better optimize their website (how?),they will continue using heavy handed tactics like blocking scrapers.
They say they're out of the country on a mission trip and are OK renting the place out below market to a responsible person (faith preferred but not required) who will take good care of it. Then they ask for a "viewing fee" before they send you a code for the non-existent lockbox with the keys.
Source: got to the "send me a viewing fee in itunes gift cards" stage of one of these scams, once. Googled the initial email and it's a common scan template.
As someone who just jumped through all the hoops to find a new apartment, the pain is real. I think the main issue is that the incentives are misaligned from the perspective of the seeker. Where I'm currently at (the Netherlands), the market is dominated by agents. Using an agent makes sense from the perspective of the apartment owner — they will take care of managing the ads, initial screening, arranging viewings etc. Given how many people contact a single viewing, you definitely don't want them hitting your personal inbox or phone.
From the agent's perspective, it's good to drive people to their company's website rather than one of the big aggregate ones. Having more people visit it means they're more likely to do business with that company specifically. Getting people to visit your company's site is pretty straightforward: they put up ads there earlier, often so much earlier that by the time you see an ad on an aggregate site, it's already fully booked for viewings.
And that's how we end up with dozens of different company sites, each crappy in their own way, and in the middle of it all, people trying to find a home pulling their hair out, refreshing a bunch of different websites every few hours.
Housing website https://funda.nl is pretty dominant in the Netherlands. Why would you need to search many agent's websites when nearly all inventory shows up on funda.nl ?
I find it much preferable to North American sites like https://realtor.ca that think that number of baths is more important than floorspace (which to my utter amazement is often not even available).
You're right in that most listings eventually end up on sites such as Funda, Pararius etc. However, almost all offers go online on an agent website first, oftentimes considerably so. Your milage will vary of course, personally I started using agent sites after calling an ad that had been up on Pararius for 5 minutes yielded the response that the viewing list is already full. Turned out it had already been up on the agent page for a day and well-priced offers go fast in desirable areas. It is worth noting that I live in a very contested area, I would hope life is better in smaller places.
I’ve noticed this as well. That is why I’ve started scraping those agency websites and send an email whenever a new listing matching my criteria came online. I even made a tool out of it for others to use [1] (it currently crawls around 50 sites).
Same in Belgium. Agencies first email new offers to their registered users, then publish on agency websites, and only afterwards publish on immoweb.be.
There are sometimes rare jewels published directly by individuals on immoweb.be but the bulk of the offers is trash.
I can't escape the feeling that if an apartment ends up being bought by a person who learns about it hours or even a few days ahead of others, then it was priced too low and the seller left money on the table.
+1 to https://funda.nl. In fact as a new immigrant I bought house completely through funda. They have nice tie-ups with all the regulatory/certificate providers such as construction report etc., Fantastic experience overall.
Selling though is a completely different story it's almost impossible without an agent. And they do provide a valuable service; like photo session, publishing on funda and others, taking care of house-viewing etc.,
>Given how many people contact a single viewing, you definitely don't want them hitting your personal inbox or phone.
This is something I noticed last time I was apartment hunting in Montréal. You'd see a sign or listing, call, and the number would be out of service. As if landlords were using virtual numbers/call forwarding services and just shutting it off once the unit was rented.
I might test this one day just to verify it did start with zillow, but after an interesting set of experiences* calling phone numbers for rentals on Zillow I started getting some very odd text messages about bank accounts I don't own and suspicious activity.
I don't call phone numbers on listing sites anymore, even if there's a face and an agency next to it. Better off looking up the agency, going to their website an calling that number (which in all three cases of the above, were different than the one on Zillow).
---
* by "interesting set of experiences" I mean calling the number on a listing took me to an automated menu and a robotic voice where I had to enter the street number, and then first five letters of a street name so it could "look up" the property I was inquiring about, and connect me to the right seller.
Hanging up on that nonsense immediately and calling the leasing agency's number via their website got me a human being in two rings, and the leasing agent for the unit I was interested in after about 45 seconds of hold music.
> From the agent's perspective, it's good to drive people to their company's website rather than one of the big aggregate ones
This is strange, wasn't my experience at all. Just about every house I viewed through funda was advertised by agents. And when I put up my house for sale the first thing the agent did was to publish on funda. All the agents care about is selling a house they get a hefty sum (0.3% if I remember correctly) and are totally fine advertising on funda.
As a house seeker I dealt with a bunch of agents and the experience wasn't all that bad to be honest. Mind you all this was in early 2019; from what I hear late 2020 onwards the market has gone bananas in the Netherlands and Amsterdam in particular.
In my experience in NL makelaars (brokers) serve particular niches, and I expect they can outcompete larger firms that don’t understand the neighborhood or municipal market as well, the market segment (roughly price), the kind of buyer/renter, the kind of seller/leaser, not to mention just their access to “deal flow” in their networks unrelated to their expertise.
Think it’s all related apartments and houses not really being fungible.
ESIT: Also, more directly to GP’s experience, when I was in the market in NL I found the biggest aggregator to be very useful, which is apparently different from their experience: funda.nl
Because the duration is too long (a year) and the cost is significant for all parties to let a middleman take a huge cut of this pie. For shorter duration stays, we already have the consolidation around airbnb.
Because it's a mostly unregulated market. A new competitor has minimal government regulations to satisfy in order to start and everyone and their dog can start doing this job (and believe me, I've met some interesting agents when looking for the best deals both looking for rent and renting out my property). The big centralised agencies don't really have a way to stop smaller agents offering cheaper services and serving different markets (even if they're just different neighbourhoods) is hard.
This is how capitalism works when there is no government helping monopolies to form in the name of security / justice / anti-money laundering / insert-politician-buzzword-of-the-day.
The natural end condition of an unregulated market is a monopoly. That is why we need regulation.
So, something is operating to provide implicit regulation. In this case it seems like chaos serves that role. Monopoly relies on a self-ordering property common to most market conditions.
The explanation GP provided is extremely low cost of entry. An agent doesn't even need an office. Here is my country many agents go free wheeling with a phone number and a bike. All they need is some connections with homeowners and they're off to business.
In San Francisco, when I was looking for an apartment,, I always ended up with an apartment with the worst pictures / missing things in the description on Craigslist, because they had less interest / were cheaper.
The market for rentals is never efficient and will never be! People who post to Zillow with a perfect description know they can extract more money for it.
I bought my house in the late-ish 90s before the web was really a big thing and certainly before things like Google StreetView. I spent a lot of time driving around to see places where I ended up not even stopping the car once I got there.
Today, there would be a lot more information available. However, as you suggest, I assume all that available information probably increases the efficiency of connecting a property with potential buyers/renters so you're less likely to luck into something that others just haven't found yet. (The funny thing is that the house I ended up buying turned out be being sold by someone I knew quite well at work.)
We almost didn’t stop into the house we now live in. It was #10 of 10 of open houses on a rainy Saturday and we were just looking to get a sense of things we both liked/didn’t like. This listing had terrible photos and the building had almost no updates since 1993. We walked in and in 2 minutes knew we’d want to buy the place, in part because it showed so poorly and the sellers were motivated to close. Zillow was a thing, but nowhere near as big a force as it is now.
When I saw the house I eventually bought, I had seen one property I was interested in but was on the fence about. When I saw this one I was pretty much sold. Great property. The house was old and very little had obviously been done with it for decades. And there were some things (like on the small side and one bathroom) that would probably have been showstoppers for a lot of people but weren't an issue for me.
Various things cost me a fair bit of money, effort, and angst that I'm glad I didn't know about at the time. But, at the end of the day, I got a ridiculously good deal--especially by today's standards--for a semi-rural place only about an hour out of Boston.
The worse the advertisement the better. I lived in an absolutely lovely townhouse which had no pictures posted and a description which failed to mention any of the amenities. My current house was shown with no staging and blackout curtains. They only got 3 offers in 2 weeks, in a market where most listings get t 30 offers in a week. It pays to look seriously when doing so takes more effort.
When I moved to Sunnyvale, I found a place on Craigslist where the entire description was typed in capital letters. The landlord was indeed quite eccentric, but we got along well. 2 bed 2 bath, vaulted ceilings. There was a pool that I maybe used once. It was good for 3 years, until his sister took over the family business and evicted him! Poor guy had a lot of mental health problems, but she was the insane one...
Because you can use your position to promote expensive listings with more profit. A truly functional aggregator would lower prices by creating efficiencies. It's more profitable to let the market utilize price discrimination as long as the favor is to the supply side. If there is an oversupply then the aggregator would work to raise prices and would be more profitable. Right now the most profitable renter favoring model would be an aggregator that charges renters a fee. The problem is the same people don't rent frequently enough so customer acquisition costs would likely be too high.
walk around the neighborhood you're thinking of renting in, and look around for signs advertising rentals with phone numbers on them.
call them with the bdrm count you're looking for, and if they know exactly what building you're talking about and units they have available, sometimes they'll give you the code to walk right in and check the place out.
you can use this convo to push the rate down a bit or ask for a rent concession, and since you're likely talking to the landlord/agency it'll be no-fee.
If you are in a hard market (SF) I would highly recommend stalking. Get a list of buildings you truly like. In SF I had like 10 I would kill to get into. When your about 1-2 months from your lease expiring, start stalking hard: call the managers, talk to people walking into the building, talk to the mail man - anything. Nice SF places rarely have openings. "Oh Mrs. Baskin passed away - we have will have a new listing in about a month". On it, I have my cashiers check, no site needed, my credit report printed, 4 references in hand, I even help the apartment manager carry in her groceries - and I don't mind hearing her stories. There are also a few families that own 20 or so buildings - once you get on their "good list" - you are in.
The places you get are amazing. Clear views of the SF harbor, tucked away in PacHeights, full parking, cheaper rent etc. After doing this for 15 years I knew all the managers of those 10 buildings by name. My friends that "wing it" get sooo screwed. They go through listings, trying to piece if it is a nice place, always getting their initial deposit stolen, application fees swiped, going to open apartments 20 people deep, dealing with large corporate landlords, on and on. Make friends!
> Is hunting for an apartment doomed to be this messy forever? It seems like an area that is ripe for disruption or a better option, but what would it take to actually solve this?
I hesitate to say it, but there is already a solution to this, and it has been the one that New York has had for years: you hire an agent. A good agent will sort through all of that for you, and streamline the process to the point where you can go from search to signing in under a week (if not quicker). Of course, there's the problem of bad agents, but that's true of anything. As you note, online services are often bad, as well.
In other words, I don't see this as a problem without a solution; I see it as a problem without a software automated solution, which is true for a lot of problems in life, particularly ones that involve huge numbers of messy, opinionated people with different motivations and interests.
In other countries, not in the USA, I've never once had a good experience with renting via an agent.
It seems it's in their job description to try to convince me to pay more for less. They listen carefully to my requirements and then take me to places that don't match at all.
I guess they figure since I've taken my entire afternoon with them, and since they've been friendly, I might give up on what I'm looking for and rent whatever it is they decided to shill that day.
Of all the countries I've rented in and had occasion to interact with agents, ones in Turkey behaved the most like cartoon caricatures: they use every sitcom used car salesman trick in the book. It would be funny to watch, except that they're perpetually wasting their time and mine.
This doesn't work in all US markets unfortunately, as I discovered recently when considering that very option you suggest. In some slower towns a few agents are willing to do this on the side as a source of income, but then in places like SF the home buying market is so hot, that it's comparatively bad ROI for agents to spend any time on most rentals. I'm curious how NYC manages to pull that off.
This was my experience as well in Seattle. I did a corporate relocation and they also provided a paid-for agent to find rentals.
She was pretty much useless. Most of the places she sent us didn't match our criteria very well, and quite a few were above where we wanted to spend. As each day went by and we nixxed place after place she clearly de-prioritized us in her work (I'm assuming she was working with multiple other people who were looking for places) and became less responsive.
At the end of the day I had to do my own hunt and fly up a second time to look at places over a weekend to find something.
And that's a paid agent from a large moving corp. I do wonder if a (more expensive) boutique agent would have been more effective but honestly having worked in real estate I consider them all at the same level as used car salesmen anyway.
I just spent two months apartment searching in NYC. I talked to many brokers but not one was interested in actively searching on my behalf. The market is incredibly disadvantageous to renters right now. Maybe parent is sharing experience from a different time or has broker connections I did not.
As an aside I also tried to automate my apartment hunt. The main thing that matters in NYC is time to respond. Unfortunately Zillow, StreetEasy, etc are not very easy to automate on the messaging side due to bot countermeasures. It was an incredibly time consuming, manual process. Happily found a great place though.
A lot of NYC agents also do this to supplement income. Real estate is feast-or-famine, and especially during the slower sales markets you can find that agents are highly incentivized to do a good job. But yeah, you're going to take a back-seat to the purchase transaction in a hot market, unless you make it worth their while on a dollar-per-hour basis. That's the downside of using a "regular" real estate agent.
In NYC specifically, there are agencies that make most of their business from renter-side representation. You can find the good ones easily on the usual review sites.
There are a lot of licensed real estate agents in the SF Bay Area. Very few of them get the lucrative sales listings. There are plenty of extra agents with time to work for renters if they wanted to. But most renters don't seem to want that service, and most lessors aren't accustomed to working with agents.
> that it's comparatively bad ROI for agents to spend any time on most rentals.
In Melbourne Australia, what I have seen is that rental agents are different and the real estate agents are different although they work for the same company.
This isn't strictly true. Historically many do, but increasingly many don't. It's been changing. Whether you pay for your own representation is also an entirely separate question.
As someone else in this thread noted, the landlord often hires an agent because they don't want to deal with the tidal wave of crap that comes from dealing with applicants. That person gets paid, obviously, and the landlord isn't the one paying them. That's the rate you're talking about here. These people are not incentivized to provide good service to the renter, and typically don't. They suck.
Higher-end buildings often have in-house agents (again: you're paying for this, whether you realize it or not), and these are called "no fee" buildings. But an increasing number of places don't have either, and just provide access to professional renter's agents instead of listing publicly. Or they do list publicly, and barely respond to the tidal wave of yahoos, knowing that motivated renters hire an agent. YMMV.
That solution is bad. You have no idea what you’re missing out on by using an agent. You might think you had a good agent, but unless you’re already really familiar with the market, you could have missed out on significantly better matches and you wouldn’t even know.
You know how we know this is a solvable problem? Listings for places to buy don’t have nearly as many problems.
Like I said, there are bad agents. There are also bad plumbers. There are bad doctors. There are bad housekeepers. This is not a problem unique to real estate, and tools exist to help you find reliable service professionals.
> You know how we know this is a solvable problem? Listings for places to buy don’t have nearly as many problems.
Just as OP said, there are tons of sites out there that claim to "solve" this, but don't. So no, I don't know that this is a computer-solvable problem.
Essentially. In NYC, many real estate agents will represent the renter for a service fee, just like they represent the buyer in a purchase transaction. It supplements the income that comes from the higher-value (but much less regular) money that they get from sales.
There are agencies that make most of their business in this area, and generally have better rate structures (typically a month or two of rent, flat rate). Others want to charge you on a percentage basis, which you can negotiate, and obviously changes the incentive structure. All of these are "expensive" (relative to free), but worth it depending on how you value your time.
Other people are (rightly) noting that there is risk of getting a scammy real estate agent, but that's true of any service. You have to look at reviews and trust your gut. Good real estate agents are motivated to do the right thing for you, because they see it as a marketing channel into larger sales in the future. Bad real estate agents are in it only for the immediate cash. These people behave in such obviously different ways that they're pretty easy to discriminate, in practice.
If the agent is in any way getting compensated when you sign a lease, or as a percentage of the rent, or anything like that, they are not working for you.
The only way they might be working for you is if you are paying them an hourly rate or per showing or something like that.
If you're saying that the historical "broker's fee" when renting in a place like NYC doesn't benefit you, then I agree. That broker represents the landlord, not you, and you're just being charged to make up the cost.
If you're saying that someone doesn't "work for you" unless you pay them a flat or hourly rate, then I disagree completely. All incentive structures have problems. Someone who works on contingency is incentivized to get you to sign. If they take a percentage cut, of course, they want you to sign the most expensive thing they can. Someone who works on a flat-rate basis is incentivized to do as little custom work as possible. Someone who works on an hourly basis is incentivized to drag out the proceedings. There is no perfect system.
Most of these agreements are structured where the fee is paid by you, after signing a lease, as a function of the monthly/annual rental price. The agent works for you, you just don't pay up front or a fixed cost or on an hourly basis. Does this change the incentive structure? Yes. Does it mean that they don't provide you a benefit? No. You just have to be aware of the incentives.
Just like when buying a house, an apartment broker works on behalf of the landlord or the tenant. They charge a fee which is a percentage of a year’s rent. It is negotiable.
Plugging my site, but only because it addresses some of your core complaints, and may be helpful.
Dwellsy.com has the largest inventory of any rental property site out there, is aiming for zero fraud (and has quite a few features implemented to prevent against it,) parses descriptions to more accurately represent the actual amenities (it's decent, but not perfect) and doesn't clump units in apartment buildings into one ranged price listing, as we prefer to have the units represented as individual listings.
Hopefully it'll help, but feel free to reach out directly if you run into issues.
You didn't even mention all the straight scam advertisements you have to wade through. Next time I move I am going to hire a PA to handle searching and making viewing appointments.
In really competitive markets it seems like you either get lucky, or you basically know someone in the area who can hook you by word-of-mouth.
In the ancient "before times" there were also actual rental agencies where a human would look for an appropriate apartment for you. I didn't know about them, but a local NorCal person turned me on and it was the only way to find things. They followed all of the different local listings daily and knew the area fairly well (e.g. how parking, transport, different commutes would be). It was $25/mo and ~$100/yr for them to go through the listings call the renter, if necessary. They usually also collected $100 fee, if they hooked you up (though I'm sure you could work around it) with one of their (usually several times a week email of 5-10) available recommendations and usually a $50+ fee for running a "credit check". The reason it seemed to work was that the renters trusted the rental agency to only send them real referrals, and the agency knew all about scammers and fake listings, and daily follow-up. It only worked due to the relatively small area and and high volume. Each agent was looking for ~50-100 people during a month and making several $k/mo themselves, filling them typically in a few months.
Like travel agents used to be an actual thing... but now you either do it yourself on-line, or you're going on some giant family cruise thing.
Asymmetric information works in favor of the landlord so I doubt it'll ever get better barring some legislation -- which we know will also never happen because most voters and politicians are also landlords
I think it depends on a few factors. The culture where you're searching. The laws. And the economy. And the available services. Where I'm from the law works pretty well, and there's really only one de-facto site that most people use for selling their pad, though Facebook has tried to take over some of the competition. I don't think you can get away from the stress of viewings, but IMHO it sounds like it's a lot more difficult to search where you're situated.
When I moved to San Francisco in 2001, there was a store front around Market and Church that was an agency with apartment listings printed out on the windows like a real estate office. They ran my credit check, made me a packet with copies of it for each landlord, and set me up with appointments to see three units, one of which I rented.
I think there was a fee in the $50 to $75 range, but it included the credit check and access to their listings.
> With all of the modernization and streamlining that has happened in other parts of our lives, apartment hunting remains unstructured, messy, frustrating, stressful.
If multiple candidates compete over the same apt, no searching method in the world will resolve the issue. Either you have high supply of apts, or a super intelligent city software that matches people to available apartments based on their socioeconomic and professional profiles.
i don't share this experience at all. i've used zillow and apartments.com recently (found a place through apartments.com about a year ago and just found one through zillow a few weeks ago) and had no issues. the filter options work how you'd expect, and i wasn't put in touch with any middle men - it was either the private property owner or the apartment management company who handled the leasing directly.
Had an excellent experience finding my last two places from Zillow - one in SD and one in Denver. Truly wonderfully luxury 1 beds at below market prices from landlords who had this as their only secondary property.
Tried the same thing with NYC and things went nowhere fast.
I had pretty good luck with Zillow and Craigslist all the times I've tried
There were a few outright lying apartment management companies, but I managed to look at some nice places being rented out directly by their owners and settled on a good one
Conclusion: it hasn't changed much at all.
It's still necessary to search on a myriad of sites. Most of the time, by the time you inquire, a place is gone. Or the ad was misleading and that "$2000 2BR" is actually "$2000 for the cheapest Studio we have, but we also have 2BR for $3.5K".
Finding places that match my criteria still requires careful inspection of the description. Checking a filter box for "Parking" excludes places that only mention parking in the description.
With all of the modernization and streamlining that has happened in other parts of our lives, apartment hunting remains unstructured, messy, frustrating, stressful.
Aggregators like Padmapper helped for awhile, but these days, it seems like most listings are posted by scammy middlemen just trying to make a commission on the listing, and now the site actively makes the process even harder.
Is hunting for an apartment doomed to be this messy forever? It seems like an area that is ripe for disruption or a better option, but what would it take to actually solve this?
---
With all of that said, this post is exactly what I needed right now. I've been toying with a very similar idea to improve the efficiency of my search process and this is good inspiration.