Database Migration Links
File Transfer Accelerators/File Copy Hints and Tips:
Migrating databases means there will be a requirement for copying
database files or export dumps from one location to another. Typically,
network speed between the two locations will be configured for normal
business workloads, not for the huge database file lifts that migration
projects can require. We therefore may need to identify ways to make the
most of the network capacity we have. In very extreme cases it may require copying files to tapes and physically transferring them to the target location, but usually we can get the performance we need either by multithreading our file copy jobs using standard file copy utilities, or by adopting a File Transfer Accelerator (FTA) utility.
The need for FTAs arises because standard TCP-based utilities such as sftp, ssh, scp or whatever don't work efficiently. Processes can end up spending more time waiting for acknowledgements
than sending data due to the way they work - TCP is serial in nature and it wont send the next set of data until the previous one is completed, including resolution of lost packets, hence the wait time. The FTAs that are available are based on the UDP protocol, rather than TCP. UDP isnt as robust as TCP in terms of managing packet loss, so the point of the FTAs is to provide that missing functionality in a way that avoids the latency issues that TCP experiences. The FTAs also include appropriate Security and other functionality to provide a complete and robust file transfer solution.
In an otherwise unloaded network link I have witnessed transfer rates at very close to the theoretical line speed, thus maximising the throughput that is possible for a given line. For example, for a 100 mbps line (45 GB/h) I have seen transfer rates above 40 GB/h using FTA utilities, compared to anything from maybe 2-12 GB/h using standard TCP-based utilities.
FTAs I have encountered include these...
Aspera : I have used Aspera
File Transfer Accelerator tools on two projects. They are cheap,
flexible, easy to install, easy to use and achieve considerable
improvements of data throughput across networks compared to standard
: FileCatalyst sounds similar to Aspera, but I havent used it to date
so have no experience to base any opinion on. The linked page on the
FileCatalyst website does, however, provide a useful summary of why
using ftp and TCP can be slow.
Linux Tip: Super-Fast Network File Copy : Some tips on using standard Linux utilities like netcat and ssh to optimise throughput of file copies
Tsunami UDP - Tsunami UDP is an open source file transfer protocol which is suggested by Amazon Web Services to use when transferring files to the AWS Cloud. I havent had an opportunity to test it, but it looks similar to Aspera and FileCatalyst, so presumably offers similar functionality and performance to tools those, but with a lower price tag!
Oracle Datapump Links:
Database Migration Resources:
EMC - Planning a Data Centre Migration - Good article covering some of the key success factors for relocating a large data center to a new facility.
Technical-Communicator.com - Documentation for Data Centre Migrations
SearchITChannel - Three steps to a successful data center migration plan
InfoSys - Successful Data Center Migration - eBook covering an end to end approach for Data Center migrations
Migrating to the Cloud - Technet Article on Oracle.com. The title refers to the Cloud, but the article itself is pretty reticent about it and the theme is really "stuff to consider when migrating from non-Oracle to Oracle". Nevertheless, it offers a pretty good high-level summary of how to undertake such migrations.
Amazon RDS - Amazon Relational Database Service (Amazon RDS) is a web service that
makes it easy to set up, operate and scale a relational database in the
cloud. This link describes the technical options currently available to migrate to Amazon RDS.
Data Migration and ETL Tool Links
Migration to Oracle from Non-Oracle
Oracle Migration Path - Oracle Partner Network Article showing the recommended database migration path to follow
when using the services available from the Oracle
Platform Migrations Group (PMG). Includes numerous links to other resources and information sources, for example...
ISV - Migrating to Oracle 9i/10g - Oracle slide deck talking about migration from non-Oracle to Oracle and the Oracel Migration Workbench
Third-Party Migration Providers - Oracle's preferred third-party migration providers offering consulting in a variety of
locations for a fixed price or on a time and materials basis