@rpathを使わずにリンクされてるから、それは効かないですね。
$ otool -L main main: /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1) /System/Library/Frameworks/UIKit.framework/UIKit (compatibility version 1.0.0, current version 61000.0.0) @rpath/libswiftCore.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftCoreFoundation.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftCoreGraphics.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftCoreImage.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftDarwin.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftDispatch.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftFoundation.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftMetal.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftObjectiveC.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftQuartzCore.dylib (compatibility version 1.0.0, current version 1001.0.82) @rpath/libswiftUIKit.dylib (compatibility version 1.0.0, current version 1001.0.82)