John Davidson

php - Web scraping - Data repeats in long polling

0 comments
Message:


I'm doing something that involves long polling. What it basically does is scrape the data from the cricket website and post it in the telegram group. I also coded a mechanism that posts the score in the group only when there is a change in the score or over. Everything works perfectly except that score repeats sometimes for no reason. What could be the possible reason for it, is it because of scraping? but it happens only sometimes


Something like this


5 score,
2 score,
5 score (repeated)


Here is my code


<?php



include 'simple_html_dom.php';

$token = 'xyz';
$group_name = 'xyz';

$cricket = file_get_html(url); // scrape data from cricket website

$score = $cricket->find('.team-score',0)->plaintext;

$old_score = file_get_contents('old_score.txt');


if($score != $old_score){
$bot = "https://api.telegram.org/bot{$token}/sendMessage?chat_id={$group_name}&text={$score}";
$hit = file_get_contents($bot);

$old_score = file_put_contents("old_score.txt",$score);
}


?>

Ajax // this would check the above file after every 1 second (long polling)


      <script type="text/javascript">
$(document).ready(function(){
setInterval(function(){
$.ajax({
url: 'score.php',
type: 'GET',
success: function(data){
console.log('success');
}
});
}, 1000);
});
</script>

repetition of data screenshot



My Answer:
It is possible that the data is repeating due to the way you are scraping the website. One possible reason for this could be that the website is updating the score multiple times in quick succession, causing your scraper to pick up the same score multiple times.

To prevent this from happening, you can try adding a check in your code to compare the current score with the previous score that was posted in the telegram group. If the scores are the same, then you can skip posting the score again.

Here is an example of how you can modify your code to include this check:

php
// Previous score variable
$prevScore = '';

// Scraping code
// Check if the current score is different from the previous score
if ($currentScore != $prevScore) {
// Post the score in the telegram group
// Your code to post the score goes here

// Update the previous score variable
$prevScore = $currentScore;
}


By adding this check, you can ensure that the score is only posted in the telegram group when it has actually changed, reducing the chances of the score repeating unnecessarily.

Rate this post

4 of 5 based on 6854 votes

Comments




© 2024 Hayatsk.info - Personal Blogs Platform. All Rights Reserved.
Create blog  |  Privacy Policy  |  Terms & Conditions  |  Contact Us