Collecting Data from the Web – Is it legal?

Over the past few weeks we’ve noticed a lot of chatter about the legality of scrubbing the web for data. This issue has been shrouded in too much ambiguity and confusion, so we thought it was about time to clear everything up – at least when it comes to how our team works.

What is web scraping?

Web scraping is the process of gathering data from websites, usually via a program that emulates the human behavior of finding information on the web. It’s useful because it helps translate a lot of unstructured data that’s available on the web into useful information that can be stored and/or analyzed.

Here’s why we’re all a little confused

We exist in the age of social media and, as a result, more and more people are choosing to make information about themselves available on social media platforms. These platforms – e.g. LinkedIn, Facebook etc. – typically require you to register and log in before accessing the data they contain. Their data is restricted, and logging in to export data from these platforms is usually a violation of their terms of service.

So far so simple.

The confusion tends to arise when you consider that, in some cases, certain social media profiles are in fact available via search engines – whether you’re logged into the platform in question or not. This is when people become – understandably – concerned that the data, despite its visibility, is also restricted.

Here’s how we can clear things up

When we first began developing our profile service, we knew the public web represented a vast resource of data that could prove invaluable to our customers. We also knew to effectively use that resource we had to fully understand all the issues involved in gathering the data. So we put together a team to conduct extensive research into all the nitty gritty details so we could become fully educated on the issues at hand.

And, here’s why you don’t have to worry

The services we provide have been developed considering every detail that our team discovered. We’ve worked very hard to ensure our methods are 100% legitimate by focusing entirely on the “public web.”

So what does that mean exactly? When we say we pull information from the “public web” we mean the part of the web which is indexed and made available via search engines – the data that is readily available without registering for or logging into any websites. In the specific case of our profile service, we are referring to the data individual users of social media platforms have actively chosen to make public.

This choice makes the entire issue wholly transparent. When a user chooses to make their profile public, they are deciding not to restrict their data to the platform they are using, choosing instead to share it with the wider web via search engines.

Once the data is made public in this way, we have the right to gather it without ever approaching any legal “gray areas” – thus ensuring you can use our services without any concerns at all.

 

Social Media: The Growing Limitations

From Social Media to Relationship MappingSocial media has changed our lives. It’s changed how we connect to our friends, our colleagues, our family, and even to consumers. But as it becomes an integral part of maintaining relationships, are we discovering its growing limitations?

Our belief in the inherent value of relationships is a key element in the rise of social media. That same belief also leads us to the inevitable discovery of its weaknesses.

If you’re a believer in the value of relationships, you might have found yourself using various forms of social media and professional profile sites (e.g., LinkedIn) to attempt to manage and grow your own relationships. You will no doubt have discovered that these services help you manage who you know as well as help you grow your network by identifying new connections.

However, in the process you may have discovered that social media struggles to deliver the ability to actually capitalize on those relationships in a professional arena – whether you’re looking for an introduction to a prospect or you just want to get a foot in the door somewhere.

So if you’re an entrepreneur, fundraiser, or sales and/or marketing professional, you’ve probably realized by now that managing your relationships through social media is not enough. What you’re missing is the ability to use your existing relationships to find connections to your targets.

So, what you’re really looking for is a Relationship Mapping service – which can connect the dots to help you do just that.