{"id":830,"date":"2010-10-23T13:48:42","date_gmt":"2010-10-23T20:48:42","guid":{"rendered":"http:\/\/briantroy.com\/blog\/?p=830"},"modified":"2010-10-23T13:48:42","modified_gmt":"2010-10-23T20:48:42","slug":"big-data-spike-volume","status":"publish","type":"post","link":"https:\/\/blogarchive.briantroy.com\/index.php\/2010\/10\/23\/big-data-spike-volume\/","title":{"rendered":"Big Data &#8211; Spike Volume"},"content":{"rendered":"<p>One aspect of Big Data is the arrival pattern of the data being stored\/analyzed.<\/p>\n<p>Yesterdays Rangers win in the ALCS is a excellent example of this. At justSignal we monitor all of the MLB teams and when the Rangers won we experienced a phenomenal spike in data arrival. Right now &#8211; which is fairly representative of normal volumes &#8211; we are receiving 7.87 Tweets per Second from Twitter. Last night we peaked out at 43.5 Tweets per Second.<\/p>\n<p>Here is what that looks like graphically:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" style=\"border:0 initial initial;\" title=\"Yankees Loss ALCS Tweets \" src=\"http:\/\/briantroy.com\/wp-content\/uploads\/2010\/10\/screen-shot-2010-10-23-at-1-37-15-pm.png\" border=\"0\" alt=\"Yankees Loss ALCS Tweets\" width=\"600\" height=\"205\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" style=\"border:0 initial initial;\" title=\"Texas Rangers ALCS Win Tweets\" src=\"http:\/\/briantroy.com\/wp-content\/uploads\/2010\/10\/screen-shot-2010-10-23-at-1-36-57-pm.png\" border=\"0\" alt=\"Texas Rangers ALCS Win Tweets\" width=\"600\" height=\"200\" \/><\/p>\n<p>Managing that kind of data arrival while continuing to generate meta data dimensions (transformations) and deliver data to clients in real time is no small task.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>One aspect of Big Data is the arrival pattern of the data being stored\/analyzed. Yesterdays Rangers win in the ALCS is a excellent example of this. At justSignal we monitor all of the MLB teams and when the Rangers won we experienced a phenomenal spike in data arrival. Right now &#8211; which is fairly representative&hellip; <a class=\"more-link\" href=\"https:\/\/blogarchive.briantroy.com\/index.php\/2010\/10\/23\/big-data-spike-volume\/\">Continue reading <span class=\"screen-reader-text\">Big Data &#8211; Spike Volume<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[28,1],"tags":[59,267],"_links":{"self":[{"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/posts\/830"}],"collection":[{"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/comments?post=830"}],"version-history":[{"count":0,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/posts\/830\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/media?parent=830"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/categories?post=830"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/tags?post=830"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}