Need Help With Large Data File Transfers - Techist - Tech Forum

Go Back   Techist - Tech Forum > Internet > Computer Networking and Internet Hardware
Click Here to Login
Reply
 
Thread Tools Display Modes
 
Old 04-03-2019, 03:12 PM   #1 (permalink)
Newb Techie
 
Join Date: Apr 2019
Location: Canada
Posts: 3
Default Need Help With Large Data File Transfers

I'm not a techie so let me get that out of the way first. I work for a start-up company that is trying to find the best way for farmers with slow internet speeds, to access large data files (500 mb) without having to wait for hours to get their data. We are currently sending them a link to OneDrive to access their imagery data.

There may not be an easy solution to this issue out there, but if there is I would sure appreciate some feedback on how this might be achieved.

The files are imagery files (.kmz or .tiff). Can they be compressed? Is there a cloud based server that can speed things up somehow? Any other suggestions??

Thanks!
__________________

__________________
cessna206 is offline   Reply With Quote
Old 04-03-2019, 05:37 PM   #2 (permalink)
True Techie
 
U.F.O.'s Avatar
 
Join Date: Mar 2019
Location: USA
Posts: 221
Default Re: Need Help With Large Data File Transfers

You need faster isp if there is one in range.

Or make wisp or mesh network, but that can be a large cost up front.

I suffered that way for a long time, eventually moved and now getting 30mpbs for $30/month and no speed throttle or cap.

On the farm it was $80 for "unlimited" lte that slowed to 3g speed for the whole month after the 10gb or whatever pittance they have at usable speed. That or sat or dial up. And some places are dark because the military jams all cell and landlines for security and they are too close to that, but to far from civilization for the cable or fiber companies to roll down the street for less than six figures.

Vice did a great expose about it: The Digital Divide

And it was about how most city dwellers or suburbanites are unaware of the lack of access in the rural community, how hard it is to raise children, pay bills, do business at competitive speed, etc and how the way the cell infrastructure was built caters to cities and there are dead zones in between, that have monopoly or nothing.

We can do better. My farm was less that quarter mile from 100mbps cable, but city attorney made franchise agreement with cable and fiber saying they are only obligated to provide resources to 70% of the city.

It was working in the 1970's when originally agreed upon, but for the past 15 years they made my community paupers and I had to leave to survive.
__________________

__________________
U.F.O. is offline   Reply With Quote
Old 04-04-2019, 07:51 AM   #3 (permalink)
Build Guru
 
PP Mguire's Avatar
 
Join Date: Dec 2004
Location: Fort Worth, Texas
Posts: 29,965
Default Re: Need Help With Large Data File Transfers

Being in Canada might make it a bit tougher depending on where the areas are. The only thing you can do is search for a new ISP for each farmer to see what's available outside of what they currently have.
__________________
"Resolution is just a number." #Ubisoft
Origin/Steam = PP_Mguire Twitch = pp_mguire Instagram = ppmguire PSN = PP_Mguire

Access to my Plex PM me.
PP Mguire is offline   Reply With Quote
Old 04-04-2019, 01:57 PM   #4 (permalink)
Newb Techie
 
limfx's Avatar
 
Join Date: Apr 2019
Location: Iraq
Posts: 3
Default Re: Need Help With Large Data File Transfers

change the resolution or make format change is possible or not
__________________
limfx is offline   Reply With Quote
Old 04-04-2019, 02:51 PM   #5 (permalink)
True Techie
 
kalju's Avatar
 
Join Date: Mar 2016
Location: EU
Posts: 113
Default Re: Need Help With Large Data File Transfers

Absolutely agree with @limfx, why should You use such a weird formats, if internet is slow. First convert all files as small format as possible, ie use such a formats, what gives enough good quality and smallest possible file size. Sometimes You can get tens and hundreds times smaller size (for example image files .bmp and jpg).
Sorry, I do not know what exactly are these .kmz or .tiff files, but it is certainly possible to convert them to other formats so that the quality is maintained, but the file size would be smaller.
Internet speed is often impossible to change. Or it requires a lot of time or resource.
And of course, you can compress them (these files), but not to a great extent.
__________________
kalju is offline   Reply With Quote
Old 04-04-2019, 03:33 PM   #6 (permalink)
Private Joker
 
carnageX's Avatar
 
Join Date: Feb 2007
Location: South Dakota
Posts: 24,840
Default Re: Need Help With Large Data File Transfers

Sounds like they're map images (going by the KMZ file extension you mentioned). @limfx & @kalju - map images tend to require very high resolution imagery (at least from my experience working with things like ArcGIS imagery data).

So going off of what PP said...they'll need better ISP's if you're wanting to push that much data. Unless you can stream the data with something like Esri's ArcGIS products, if mapping is indeed what you're doing.
__________________
Laptop: MSI GT70 2OC-059us | i7-4700MQ | 16GB | GTX 770m | 500GB SSD / 750GB HDD | 17.3" | Win10 Pro
Desktop: 4690k | 12GB g.Skill RipJaws | GTX 970 | 520hx | Z87X-UD4H | Corsair Vengeance C70 | Corsair H110 | Acer 25" | Acer 22" | Win10
Mobile: Google Pixel XL


If I help you, or you just like what I said, rep me by clicking the under my post
carnageX is offline   Reply With Quote
Old 04-04-2019, 04:26 PM   #7 (permalink)
Grandfather of Techist

¯\_(ツ)_/¯
 
Trotter's Avatar
 
Join Date: Jan 2005
Location: The South
Posts: 31,593
Default Re: Need Help With Large Data File Transfers

Not trying to sound all Luddite but have you considered putting them on small and dirt cheap flash drives and sending them to said farmers? Sometimes simple solutions work the best.
__________________
Cougar MX330-G / Antec EA650 650W / Zotac GTX 1660 AMP
AMD Ryzen 5 1600 / MSI B350 Gaming Plus / Samsung 970 EVO 250GB M.2 SSD
WD Blue SSD 250GB / WD Black 750GB / WD Green 1TB / 2x16GB Corsair Vengence RGB
Win10 Pro x64 / Razer DeathAdder Elite / Tecware 87 Key Mechanical

R.I.P. Father Trotter ... 14 Nov 1945 - 4 Sept 2009
Trotter is offline   Reply With Quote
Old 04-04-2019, 09:38 PM   #8 (permalink)
True Techie
 
U.F.O.'s Avatar
 
Join Date: Mar 2019
Location: USA
Posts: 221
Default Re: Need Help With Large Data File Transfers

Sneaker net was my first response too but I was too ashamed to suggest it as first reply. Meh it works. You could get a lot of usb flash in bulk and go to the mailbox a lot haha. Lots of gps apps use kmz I don't think it's weird. Maybe dude runs a survey company?
__________________
U.F.O. is offline   Reply With Quote
Old 04-05-2019, 07:48 AM   #9 (permalink)
Build Guru
 
PP Mguire's Avatar
 
Join Date: Dec 2004
Location: Fort Worth, Texas
Posts: 29,965
Default Re: Need Help With Large Data File Transfers

I didn't suggest that because I have a feeling it's updated map content based on agricultural growth.
__________________
"Resolution is just a number." #Ubisoft
Origin/Steam = PP_Mguire Twitch = pp_mguire Instagram = ppmguire PSN = PP_Mguire

Access to my Plex PM me.
PP Mguire is offline   Reply With Quote
Old 04-05-2019, 08:11 AM   #10 (permalink)
Private Joker
 
carnageX's Avatar
 
Join Date: Feb 2007
Location: South Dakota
Posts: 24,840
Default Re: Need Help With Large Data File Transfers

Quote:
Originally Posted by PP Mguire View Post
I didn't suggest that because I have a feeling it's updated map content based on agricultural growth.
My thoughts as well.

Or even flooding, property boundaries, hunting lands, etc. I know we provide info like that to state citizens here - and then some - through our services we map through Esri.
__________________

__________________
Laptop: MSI GT70 2OC-059us | i7-4700MQ | 16GB | GTX 770m | 500GB SSD / 750GB HDD | 17.3" | Win10 Pro
Desktop: 4690k | 12GB g.Skill RipJaws | GTX 970 | 520hx | Z87X-UD4H | Corsair Vengeance C70 | Corsair H110 | Acer 25" | Acer 22" | Win10
Mobile: Google Pixel XL


If I help you, or you just like what I said, rep me by clicking the under my post
carnageX is offline   Reply With Quote
Reply

Tags
kmz, large data, tiff

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Study Suggests That Google Toolbar Transfers Data Even In Disabled State Osiris Tips, Tricks & Tutorials 1 01-26-2010 12:21 PM
Pod To PC Transfers Data From iPods And iPhone To PC Osiris Tips, Tricks & Tutorials 0 02-10-2009 07:08 AM
Slow in-network file transfers dj2501 Computer Networking and Internet Hardware 3 03-26-2007 10:59 PM
Problem: cracking/popping when data transfers mookie3333 Computer Audio and Multimedia 5 04-06-2005 04:50 PM
Cmd.exe FTP get cmd, transfers remote file to what local dir? Ropponmatsu Computer Networking and Internet Hardware 5 01-01-2004 12:47 PM



Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -5. The time now is 02:25 PM.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2019, vBulletin Solutions, Inc.