Prolific Reviews & Product Details

I use prolific many times a year to run research studies. It is easy to use for beginners – you can even run short studies within prolific. I have implemented 99 studies over 7 years of useage. But a major strength is in linking to other systems, like Qualtrics. Customer support is there if you need it, along with a helpful community. hihgly recommended: The data quality is second to none, in my experience, and the security and payments handling has been faultless. Review collected by and hosted on G2.com.

There are not many dislikes. Minor UI issues, like sorting previous studies, or approving participants are often sorted in response to user feedback. I guess the one niggle is price, but it is good value for money with few competitors. It would be nice to be able to recruit larger samples outside the UK/US, but that is coming along also. Review collected by and hosted on G2.com.

I like that you can customise the screeners, even down to excluding individual participants (for example, those who have completed previous experiments of yours or your colleagues). Data collection is very quick and the participants are highly motivated to complete to a high quality. It is a million times easier than recruiting participants in person and you can highly customise your participant pool, making it a representative sample of the population too. It is simple and easy to integrate with your experiment platform. I've used Prolific several times now for studies and am overall very happy with it. Review collected by and hosted on G2.com.

Sometimes participants have clearly lied about things like their language proficiency, or have not read the instructions that state that they must be from a certain region of the UK, and they get very cross if you reject them on this basis and insist on payment. Customer support could sometimes do with being quicker, as sometimes a participants payment has been approved before they have had a chance to return a submission for you. Review collected by and hosted on G2.com.

1) the ease of use, 2) the efficiency of collecting data, 3) the diversity of sample population, 4) the quality of partcipants Review collected by and hosted on G2.com.

I feel that Prolific places significant emphasis on protecting participants' interests, which is commendable. However, I have noticed that sometimes participants provide invalid or careless responses to studies or surveys. Despite this, I feel compelled to approve their submissions due to concerns about fairness and Prolific’s policies. This creates a situation where some participants can complete studies extremely quickly and carelessly without facing any consequences.

It would be beneficial if Prolific established clearer rules or guidelines to ensure that participants adhere to standards of quality and provide thoughtful and careful responses when completing studies. Review collected by and hosted on G2.com.

Prolific makes finding participants with specific demographic backgrounds extremely easy. Moreover, once you've posted an experiment, data collection proceeds extremely quickly. Setting up a study is straightforward, and is easy to integrate with other internet-based data collection platforms. In the rare cases where I've encountered issues (e.g. with generating an invoice for study payments), I've had quick and useful feedback from customer support. Finally, the quality of the data collected tends to be pretty good (as compared to some competitor platforms, e.g. Amazon Mechanical Turk). Review collected by and hosted on G2.com.

For all that it is straightforward to filter for specific demographic qualifications, the coverage of the user-base doesn't always align with our needs. Recruitment from English-speaking populations is fantastic, but it is often much more difficult to find participants from other language communities (e.g. German). In addition, the service fee is steep. This means that I typically only use prolific if I am in a hurry, or cannot otherwise find participants in my local research network. Review collected by and hosted on G2.com.

The process to recruit participants is easy. Review collected by and hosted on G2.com.

More than half of my participants significantly increase the duration spent on the task by either staying idle on the consent form or after finishing the task. I suspect this is because they expect an increase in rewards, as many complain about how I underestimated the time and demand more rewards. However, I recruited hundreds of participants, and it only occurred twice when a subject spent significantly more time on the task than the average.

What participants log on Prolific does not reflect the actual time spent on the task. I record reaction times. However, Prolific displays the average (Prolific) time for each experiment and pressures researchers to increase the pay, even when participants deliberately manipulate the average duration.

I collected different batches of data for the exact same experiment and often noticed the following behavior: If my first one or two participants complete the task quickly, most of the subsequent participants follow suit. Conversely, if my first few participants take significantly longer than the estimated time, the rest do the same. Perhaps they are influenced by the power of the crowd; the more participants exceed the time limit, the more likely an increase in rewards will occur. That is why I find publicly displaying the average (Prolific) duration problematic.

The inaccurate, publicly displayed estimated time duration provided by Prolific has made collecting data on the platform quite stressful. I receive many complaints, even though they are not valid.

I also think Prolific should not automatically approve submissions. They should either require researchers to allow 7 days for participants to return their submissions and remove the automatic submission feature or not require a 7-day waiting period and automatically approve submissions. I do not want to reject participants' submissions as I know it affects their approval rate. However, many participants take advantage of this and do not return their submissions, hoping that the researcher will forget about it and it will get approved automatically. Indeed, being busy, I often forget about those participants that I did not want to reject. Their submissions get approved without providing any useful data.

Prolific is also unreliable in terms of support. Sometimes, it takes them several weeks to respond to my requests. On several occasions, I requested a special invoice, and they continued to make mistakes on it, leading me to wonder if they even read the requests. Additionally, I received each response one week apart. Considering they take 25–30% of research funds, Prolific needs to demonstrate greater professionalism. Review collected by and hosted on G2.com.

Filtering for specific characteristics of participants is very clear. They do a good job of ensuring that you are honest and ethical with your pay to the participants, which I think helps keep the data at a higher quality than other platforms. Setting up studies is very straightforward; they outline clear guidelines for how to comply with their requirements. Customer Support responds quickly. Review collected by and hosted on G2.com.

Obviously, they aren't going to have every specific characteristic for users, and extremely specific samples might need to look elsewhere (but it is worth checking to see if they do have it). Sometimes, I have found discrepancies between their demographic data of the participants and what the participants self-report (especially in terms of political beliefs), but I think that's endemic to every platform. Review collected by and hosted on G2.com.

The assurance of anonymity.

The speed of setting up a study, and the speed of getting results, so you can do a quick pilot study to test out an idea with ease and relatively low cost.

The ability to screen participants so you can target your recruitment efforts

The ability to follow up with participants for further data collection Review collected by and hosted on G2.com.

Sometimes participants do not provide quality data, and rush through the survey. If you withhold payment because of this they can get quite agitated. Review collected by and hosted on G2.com.

the participants that complete my study on prolific put in good effort and there are few who fail attention checks. Also very divers group of participants Review collected by and hosted on G2.com.

I would like it to be easier to make studies with participants from different regions of the world (in a more general sense than country by country). Additionally, I would like it to be easier to rerun a study with more people. Review collected by and hosted on G2.com.

I like how easy it is to screen participants and how quickly one can collect data. Payment is fast and easy too, and now that I can recruit using multiple URLs, implementing most of my experiments has turned into a breeze, especially those with multiple stimuli sets. Integrating my experiment into the Prolific workflow is trivial. It's the only platform I use for data collection. Nothing compares! Review collected by and hosted on G2.com.

I find the messaging interface to be a bit clunky, and I wish it would show which devices you have chosen to allow in the summary before deployment. I also find the custom screeners to be hard to use. Review collected by and hosted on G2.com.

Prolific is easy to use, with a mostly well-thought-out interface and an excellent feature set. The support is helpful and responsive. Prices are fair and reasonable. There is also a powerful API which most users won't need but it does make some interesting advanced use cases possible, which we have used in our organization. Review collected by and hosted on G2.com.

Sometimes the interface is very slow, but this is just an inconvenience. Review collected by and hosted on G2.com.