BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20181221T160904Z
LOCATION:C2/3/4 Ballroom
DTSTART;TZID=America/Chicago:20181113T083000
DTEND;TZID=America/Chicago:20181113T170000
UID:submissions.supercomputing.org_SC18_sess322_post140@linklings.com
SUMMARY:Large-Message Size Allreduce at Wire Speed for Distributed Deep Le
 arning
DESCRIPTION:Poster\nTech Program Reg Pass, Exhibits Reg Pass\n\nLarge-Mess
 age Size Allreduce at Wire Speed for Distributed Deep Learning\n\nTanaka, 
 Arikawa, Kawai, Kato, Ito...\n\nIn large-scale distributed deep learning, 
 the Allreduce operation for large messages (100 KB or more) is critical fo
 r gathering gradients from multiple worker nodes and broadcasting the sum 
 of the gradients to them. When the message is large, the latency in Allred
 uce operation would make it difficult to take advantage of large-scale dis
 tributed deep learning. To reduce the latency, we devised a dataflow archi
 tecture with an Allreduce-specific hardware accelerator that performs data
  aggregation and reduction while data is being transferred. The accelerato
 r is designed to immediately start Allreduce operation before an entire me
 ssage is recived. Furthermore, Allreduce can be operated at wire speed by 
 vectorizing the gradients and summing them in parallel. Experimental resul
 ts reveal that the proposed architecture performs Allreduce at 96% of wire
  speed for a large message. Moreover, the latency of Allreduce is reduced 
 by 65% compared with a state-of-the-art Allreduce method when applied for 
 ResNet-50.
URL:https://sc18.supercomputing.org/presentation/?id=post140&sess=sess322
END:VEVENT
END:VCALENDAR