blallo
/
Signal-Android
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Add ringrtc support 

Initial commit of the RingRTC Java interface implementation.

The implementation lives in an external .aar with the package
org.signal.ringrtc.

The package provides two high level objects of interest
=======================================================

org.signal.ringrtc.CallConnection -- represents the session of a call,
very similar to WebRTC's PeerConnection.

org.signal.ringrtc.CallConnectionFactory -- creates CallConnection
objects, very similar to WebRTC's PeerConnectionFactory.

The implementation interfaces with the Android application in a few
places:
==================================================================

src/org/thoughtcrime/securesms/ApplicationContext.java -- RingRTC
library initialization at application startup.

src/org/thoughtcrime/securesms/service/WebRtcCallService.java -- Call
creation and state machine.

src/org/thoughtcrime/securesms/ringrtc -- this package implements
interface classes needed by ringrtc and a CallConnectionWrapper helper
class.

The two interfaces needed so far are:

  ringrtc/Logger.java
  ringrtc/SignalMessageRecipient.java

The logger is self-explanatory, but SignalMessageRecipient is a little
more involved.  SignalMessageRecipient encapsulates the Signal-Android
notion of "Recipient" and the mechanism for sending Signal Messages
related to audio/video calling.

The CallConnectionWrapper class is clone of the original
org.thoughtcrime.securesms.webrtc.PeerConnectionWrapper, suitably
modified to match the CallConnection interface.

This class continues to handle the Camera switching APIs, with that
portion of the code remaining unmodified from the original.

CallConnectionFactory Details
=============================

The primary public methods:

initialize() -- initialize the WebRTC library and RingRTC library.
The WebRTC initialization is lifted from the original Signal-Android
code.

createCallConnectionFactory() -- creates a CallConnectionFactory
object.  Internally it creates a WebRTC PeerConnectionFactory object
and a RingRTC CallConnectionFactory object.

dispose() -- tears down the CallConnectionFactory object, including
the internal PeerConnectionFactory and RingRTC CallConnectionFactory.

createCallConnection() -- creates a CallConnection object, connecting
that with an application controlled CallConnection.Observer object.

This function takes a CallConnection.Configuration object to link the
CallConnection object with some application provided services, like
sending Signal protocol messages.

CallConnection Details
======================

This object is a subclass of WebRTC's PeerConnection class.

The primary public methods and objects:

CallConnection.Configuration
----------------------------

Configuration object used to parameterize a call.  Notable members:

- SignalServiceMessageSender messageSender
- long callId
- org.signal.SignalMessageRecipient recipient

The 'accountManager' is used to fetch public information from the Signal
service, specifically used here to obtain the public Signal TURN
server details.

The 'callId' is a 64-bit pseudo-random number generated when the call
is initiated, used to identify the call through out its lifetime.

The "recipient' is an implementation of the
org.signal.SignalMessageRecipient interface, which encapsulates the
sending of Signal service messages to a recipient (remote peer) using
existing Signal protocol data structures.

The native library needs to be able to send Signal messages via the
service, but it does not have a native implementation to do so.
Instead the native code calls out to the client for sending Signal
messages.  To accomplish this, the client implements the
org.signal.SignalMessageRecipient interface and passes an instance of
that in a CallConnection.Configuration object.

CallConnection
--------------

dispose() -- tears down the CallConnection object, including the
internal PeerConnection and RingRTC CallConnection.

sendOffer() -- initiates a call to a remote recipient.  This is the
beginning of an outbound call.

validateResponse() -- checks an offer response recipient against the
originating call details.

handleOfferAnswer() -- handles the receipt of answer, which was a
response from an originating offer.

acceptOffer() -- accept an offer from a remote participant.  This is
the begin of an incoming call.

answerCall() -- invoked when the call is completely established and
online.

hangUp() -- hang up the connection and shut things done.  This is the
end of the call.

sendBusy() -- send the remote side an indication that the local side
is already in a call and the line is busy.

sendVideoStatus() -- send the current state of the local camera video
stream to the remote side.

CallConnection.Observer
-----------------------

Observer object, used by the RingRTC library to notify the client
application of important events and status changes.  Similar in spirit
to WebRTC's PeerConnection.Observer.

Observer callbacks come in three flavors:

- state change notifications,
- on stream notifications
- errors conditions

For state notifications, the callback contains the callId, the
recipient and a CallConnection.CallEvent type.

For streams, the callback contains the callId, the
recipient and a org.webrtc.MediaStream.

For errors, the callback contains the callId, the recipient and an
exception type.  The currently thrown exceptions include:

- UntrustedIdentityException
- UnregisteredUserException
- IOException

Signed-off-by: Curt Brune <curt@signal.org>

Updates to support ringrtc-android version 0.1.0.

* simplify logging interface

It is no longer necessary for the application to specify a Log object
as the library can log via the NDK directly.

* improve error handling and notification

In a number of places where ringrtc errors could occur, no
notification was ever sent to the user, nor was the UI cleaned up.  It
would look like the app was in hung state.

This patch updates these situations to send the WebRtcViewModel a
NETWORK_FAILURE message.

* update handleIncomingCall() for lockManager and notification

During the conversion to RingRTC, the implementation of
handleIncomingCall() missed a couple of things:

-- updating the Phone state with the lockManager
-- sending a message to the viewModel

* log the callId in various handler methods

For debugging purposes it is very handy to have the callId present in
the log during the various call handler methods.

Signed-off-by: Curt Brune <curt@signal.org>
master
Curt Brune 2019-05-17 09:19:09 -07:00 committed by Greyson Parrelli
parent 37bcac40bb
commit 7f0a7b0c13
12 changed files with 788 additions and 3168 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
build.gradle
View File

@ -88,7 +88,7 @@ dependencies {
implementation 'org.whispersystems:signal-service-android:2.13.7'
implementation 'org.whispersystems:webrtc-android:M75'
implementation 'org.signal:ringrtc-android:0.1.0'
implementation "me.leolin:ShortcutBadger:1.1.16"
implementation 'se.emilsjolander:stickylistheaders:2.7.0'
@ -194,7 +194,7 @@ dependencyVerification {
'org.conscrypt:conscrypt-android:400ca559a49b860a82862b22cee0e3110764bdcf7ee7c79e7479895c25cdfc09',
'org.signal:aesgcmprovider:6eb4422e8a618b3b76cb2096a3619d251f9e27989dc68307a1e5414c3710f2d1',
'org.whispersystems:signal-service-android:5115aa434c52ca671c513995e6ae67d73f3abaaa605f9e6cf64c2e01da961c7e',
'org.whispersystems:webrtc-android:f8231bb57923afb243760213dc58924e85cce42f2f3cc8cb33a6d883672a921a',
'org.signal:ringrtc-android:f2ccac4060d04fb1010a2892ef1a6048dc85185499e3277cb4349d8d21fa37e3',
'me.leolin:ShortcutBadger:e3cb3e7625892129b0c92dd5e4bc649faffdd526d5af26d9c45ee31ff8851774',
'se.emilsjolander:stickylistheaders:a08ca948aa6b220f09d82f16bbbac395f6b78897e9eeac6a9f0b0ba755928eeb',
'com.jpardogo.materialtabstrip:library:c6ef812fba4f74be7dc4a905faa4c2908cba261a94c13d4f96d5e67e4aad4aaa',

3
proguard-webrtc.pro
View File

@ -1,3 +0,0 @@
-dontwarn org.webrtc.NetworkMonitorAutoDetect
-dontwarn android.net.Network
-keep class org.webrtc.** { *; }

9
src/org/thoughtcrime/securesms/ApplicationContext.java
View File

@ -33,6 +33,7 @@ import com.google.android.gms.security.ProviderInstaller;
import org.conscrypt.Conscrypt;
import org.signal.aesgcmprovider.AesGcmProvider;
import org.signal.ringrtc.CallConnectionFactory;
import org.thoughtcrime.securesms.components.TypingStatusRepository;
import org.thoughtcrime.securesms.components.TypingStatusSender;
import org.thoughtcrime.securesms.database.DatabaseFactory;
@ -69,8 +70,6 @@ import org.thoughtcrime.securesms.service.RotateSignedPreKeyListener;
import org.thoughtcrime.securesms.service.UpdateApkRefreshListener;
import org.thoughtcrime.securesms.util.TextSecurePreferences;
import org.thoughtcrime.securesms.util.dynamiclanguage.DynamicLanguageContextWrapper;
import org.webrtc.PeerConnectionFactory;
import org.webrtc.PeerConnectionFactory.InitializationOptions;
import org.webrtc.voiceengine.WebRtcAudioManager;
import org.webrtc.voiceengine.WebRtcAudioUtils;
import org.whispersystems.libsignal.logging.SignalProtocolLoggerProvider;
@ -125,7 +124,7 @@ public class ApplicationContext extends MultiDexApplication implements DefaultLi
initializeSignedPreKeyCheck();
initializePeriodicTasks();
initializeCircumvention();
initializeWebRtc();
initializeRingRtc();
initializePendingMessages();
initializeUnidentifiedDeliveryAbilityRefresh();
initializeBlobProvider();
@ -280,7 +279,7 @@ public class ApplicationContext extends MultiDexApplication implements DefaultLi
}
}
private void initializeWebRtc() {
private void initializeRingRtc() {
try {
Set<String> HARDWARE_AEC_BLACKLIST = new HashSet<String>() {{
add("Pixel");
@ -309,7 +308,7 @@ public class ApplicationContext extends MultiDexApplication implements DefaultLi
WebRtcAudioManager.setBlacklistDeviceForOpenSLESUsage(true);
}
PeerConnectionFactory.initialize(InitializationOptions.builder(this).createInitializationOptions());
CallConnectionFactory.initialize(this);
} catch (UnsatisfiedLinkError e) {
Log.w(TAG, e);
}

2
src/org/thoughtcrime/securesms/components/webrtc/WebRtcCallScreen.java
View File

@ -43,10 +43,10 @@ import org.thoughtcrime.securesms.R;
import org.thoughtcrime.securesms.mms.GlideApp;
import org.thoughtcrime.securesms.recipients.Recipient;
import org.thoughtcrime.securesms.recipients.RecipientModifiedListener;
import org.thoughtcrime.securesms.ringrtc.CameraState;
import org.thoughtcrime.securesms.util.Util;
import org.thoughtcrime.securesms.util.VerifySpan;
import org.thoughtcrime.securesms.util.ViewUtil;
import org.thoughtcrime.securesms.webrtc.CameraState;
import org.webrtc.SurfaceViewRenderer;
import org.whispersystems.libsignal.IdentityKey;

2
src/org/thoughtcrime/securesms/events/WebRtcViewModel.java
View File

@ -4,7 +4,7 @@ import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import org.thoughtcrime.securesms.recipients.Recipient;
import org.thoughtcrime.securesms.webrtc.CameraState;
import org.thoughtcrime.securesms.ringrtc.CameraState;
import org.webrtc.SurfaceViewRenderer;
import org.whispersystems.libsignal.IdentityKey;

293
src/org/thoughtcrime/securesms/ringrtc/CallConnectionWrapper.java 100644
View File

@ -0,0 +1,293 @@
package org.thoughtcrime.securesms.ringrtc;
import android.content.Context;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import java.io.IOException;
import org.signal.ringrtc.CallConnection;
import org.signal.ringrtc.CallConnectionFactory;
import org.signal.ringrtc.CallException;
import org.signal.ringrtc.SignalMessageRecipient;
import org.thoughtcrime.securesms.logging.Log;
import org.webrtc.AudioSource;
import org.webrtc.AudioTrack;
import org.webrtc.Camera1Enumerator;
import org.webrtc.Camera2Enumerator;
import org.webrtc.CameraEnumerator;
import org.webrtc.CameraVideoCapturer;
import org.webrtc.EglBase;
import org.webrtc.IceCandidate;
import org.webrtc.MediaConstraints;
import org.webrtc.MediaStream;
import org.webrtc.SurfaceTextureHelper;
import org.webrtc.VideoSink;
import org.webrtc.VideoSource;
import org.webrtc.VideoTrack;
import org.whispersystems.signalservice.api.SignalServiceAccountManager;
import org.whispersystems.signalservice.api.push.exceptions.UnregisteredUserException;
import static org.thoughtcrime.securesms.ringrtc.CameraState.Direction.BACK;
import static org.thoughtcrime.securesms.ringrtc.CameraState.Direction.FRONT;
import static org.thoughtcrime.securesms.ringrtc.CameraState.Direction.NONE;
import static org.thoughtcrime.securesms.ringrtc.CameraState.Direction.PENDING;
public class CallConnectionWrapper {
private static final String TAG = Log.tag(CallConnectionWrapper.class);
@NonNull private final CallConnection callConnection;
@NonNull private final AudioTrack audioTrack;
@NonNull private final AudioSource audioSource;
@NonNull private final Camera camera;
@Nullable private final VideoSource videoSource;
@Nullable private final VideoTrack videoTrack;
public CallConnectionWrapper(@NonNull Context context,
@NonNull CallConnectionFactory factory,
@NonNull CallConnection.Observer observer,
@NonNull VideoSink localRenderer,
@NonNull CameraEventListener cameraEventListener,
@NonNull EglBase eglBase,
boolean hideIp,
long callId,
boolean outBound,
@NonNull SignalMessageRecipient recipient,
@NonNull SignalServiceAccountManager accountManager)
throws UnregisteredUserException, IOException, CallException
{
CallConnection.Configuration configuration = new CallConnection.Configuration(callId,
outBound,
recipient,
accountManager,
hideIp);
this.callConnection = factory.createCallConnection(configuration, observer);
this.callConnection.setAudioPlayout(false);
this.callConnection.setAudioRecording(false);
MediaStream mediaStream = factory.createLocalMediaStream("ARDAMS");
MediaConstraints audioConstraints = new MediaConstraints();
audioConstraints.optional.add(new MediaConstraints.KeyValuePair("DtlsSrtpKeyAgreement", "true"));
this.audioSource = factory.createAudioSource(audioConstraints);
this.audioTrack = factory.createAudioTrack("ARDAMSa0", audioSource);
this.audioTrack.setEnabled(false);
mediaStream.addTrack(audioTrack);
this.camera = new Camera(context, cameraEventListener);
if (camera.capturer != null) {
this.videoSource = factory.createVideoSource(false);
this.videoTrack = factory.createVideoTrack("ARDAMSv0", videoSource);
camera.capturer.initialize(SurfaceTextureHelper.create("WebRTC-SurfaceTextureHelper", eglBase.getEglBaseContext()), context, videoSource.getCapturerObserver());
this.videoTrack.addSink(localRenderer);
this.videoTrack.setEnabled(false);
mediaStream.addTrack(videoTrack);
} else {
this.videoSource = null;
this.videoTrack = null;
}
this.callConnection.addStream(mediaStream);
}
public boolean addIceCandidate(IceCandidate candidate) {
return callConnection.addIceCandidate(candidate);
}
public void sendOffer() throws CallException {
callConnection.sendOffer();
}
public boolean validateResponse(SignalMessageRecipient recipient, Long inCallId)
throws CallException
{
return callConnection.validateResponse(recipient, inCallId);
}
public void handleOfferAnswer(String sessionDescription) throws CallException {
callConnection.handleOfferAnswer(sessionDescription);
}
public void acceptOffer(String offer) throws CallException {
callConnection.acceptOffer(offer);
}
public void hangUp() throws CallException {
callConnection.hangUp();
}
public void answerCall() throws CallException {
callConnection.answerCall();
}
public void sendBusy(SignalMessageRecipient recipient, Long inCallId) throws CallException
{
callConnection.sendBusy(recipient, inCallId);
}
public void setVideoEnabled(boolean enabled) throws CallException {
if (videoTrack != null) {
videoTrack.setEnabled(enabled);
}
camera.setEnabled(enabled);
callConnection.sendVideoStatus(enabled);
}
public void flipCamera() {
camera.flip();
}
public CameraState getCameraState() {
return new CameraState(camera.getActiveDirection(), camera.getCount());
}
public void setCommunicationMode() {
callConnection.setAudioPlayout(true);
callConnection.setAudioRecording(true);
}
public void setAudioEnabled(boolean enabled) {
audioTrack.setEnabled(enabled);
}
public void dispose() {
camera.dispose();
if (videoSource != null) {
videoSource.dispose();
}
audioSource.dispose();
callConnection.dispose();
}
private static class Camera implements CameraVideoCapturer.CameraSwitchHandler {
@Nullable
private final CameraVideoCapturer capturer;
private final CameraEventListener cameraEventListener;
private final int cameraCount;
private CameraState.Direction activeDirection;
private boolean enabled;
Camera(@NonNull Context context, @NonNull CameraEventListener cameraEventListener)
{
this.cameraEventListener = cameraEventListener;
CameraEnumerator enumerator = getCameraEnumerator(context);
cameraCount = enumerator.getDeviceNames().length;
CameraVideoCapturer capturerCandidate = createVideoCapturer(enumerator, FRONT);
if (capturerCandidate != null) {
activeDirection = FRONT;
} else {
capturerCandidate = createVideoCapturer(enumerator, BACK);
if (capturerCandidate != null) {
activeDirection = BACK;
} else {
activeDirection = NONE;
}
}
capturer = capturerCandidate;
}
void flip() {
if (capturer == null || cameraCount < 2) {
throw new AssertionError("Tried to flip the camera, but we only have " + cameraCount +
" of them.");
}
activeDirection = PENDING;
capturer.switchCamera(this);
}
void setEnabled(boolean enabled) {
this.enabled = enabled;
if (capturer == null) {
return;
}
try {
if (enabled) {
capturer.startCapture(1280, 720, 30);
} else {
capturer.stopCapture();
}
} catch (InterruptedException e) {
Log.w(TAG, "Got interrupted while trying to stop video capture", e);
}
}
void dispose() {
if (capturer != null) {
capturer.dispose();
}
}
int getCount() {
return cameraCount;
}
@NonNull CameraState.Direction getActiveDirection() {
return enabled ? activeDirection : NONE;
}
@Nullable CameraVideoCapturer getCapturer() {
return capturer;
}
private @Nullable CameraVideoCapturer createVideoCapturer(@NonNull CameraEnumerator enumerator,
@NonNull CameraState.Direction direction)
{
String[] deviceNames = enumerator.getDeviceNames();
for (String deviceName : deviceNames) {
if ((direction == FRONT && enumerator.isFrontFacing(deviceName)) ||
(direction == BACK && enumerator.isBackFacing(deviceName)))
{
return enumerator.createCapturer(deviceName, null);
}
}
return null;
}
private @NonNull CameraEnumerator getCameraEnumerator(@NonNull Context context) {
boolean camera2EnumeratorIsSupported = false;
try {
camera2EnumeratorIsSupported = Camera2Enumerator.isSupported(context);
} catch (final Throwable throwable) {
Log.w(TAG, "Camera2Enumator.isSupport() threw.", throwable);
}
Log.i(TAG, "Camera2 enumerator supported: " + camera2EnumeratorIsSupported);
return camera2EnumeratorIsSupported ? new Camera2Enumerator(context)
: new Camera1Enumerator(true);
}
@Override
public void onCameraSwitchDone(boolean isFrontFacing) {
activeDirection = isFrontFacing ? FRONT : BACK;
cameraEventListener.onCameraSwitchCompleted(new CameraState(getActiveDirection(), getCount()));
}
@Override
public void onCameraSwitchError(String errorMessage) {
Log.e(TAG, "onCameraSwitchError: " + errorMessage);
cameraEventListener.onCameraSwitchCompleted(new CameraState(getActiveDirection(), getCount()));
}
}
public interface CameraEventListener {
void onCameraSwitchCompleted(@NonNull CameraState newCameraState);
}
}

2
src/org/thoughtcrime/securesms/webrtc/CameraState.java → src/org/thoughtcrime/securesms/ringrtc/CameraState.java
View File

@ -1,4 +1,4 @@
package org.thoughtcrime.securesms.webrtc;
package org.thoughtcrime.securesms.ringrtc;
import androidx.annotation.NonNull;

115
src/org/thoughtcrime/securesms/ringrtc/MessageRecipient.java 100644
View File

@ -0,0 +1,115 @@
package org.thoughtcrime.securesms.ringrtc;
import android.content.Context;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import java.io.IOException;
import java.util.List;
import org.signal.ringrtc.SignalMessageRecipient;
import org.thoughtcrime.securesms.crypto.UnidentifiedAccessUtil;
import org.thoughtcrime.securesms.database.Address;
import org.thoughtcrime.securesms.logging.Log;
import org.thoughtcrime.securesms.recipients.Recipient;
import org.thoughtcrime.securesms.util.Util;
import org.whispersystems.signalservice.api.SignalServiceMessageSender;
import org.whispersystems.signalservice.api.crypto.UntrustedIdentityException;
import org.whispersystems.signalservice.api.push.exceptions.UnregisteredUserException;
import org.whispersystems.signalservice.api.messages.calls.AnswerMessage;
import org.whispersystems.signalservice.api.messages.calls.BusyMessage;
import org.whispersystems.signalservice.api.messages.calls.IceUpdateMessage;
import org.whispersystems.signalservice.api.messages.calls.HangupMessage;
import org.whispersystems.signalservice.api.messages.calls.OfferMessage;
import org.whispersystems.signalservice.api.messages.calls.SignalServiceCallMessage;
import org.whispersystems.signalservice.api.push.SignalServiceAddress;
public final class MessageRecipient implements SignalMessageRecipient {
private static final String TAG = Log.tag(MessageRecipient.class);
@NonNull private final Recipient recipient;
@NonNull private final SignalServiceMessageSender messageSender;
public MessageRecipient(SignalServiceMessageSender messageSender,
Recipient recipient)
{
this.recipient = recipient;
this.messageSender = messageSender;
}
public @NonNull Address getAddress() {
return recipient.getAddress();
}
@Override
public boolean isEqual(@NonNull SignalMessageRecipient o) {
if (!(o instanceof MessageRecipient)) {
return false;
}
MessageRecipient that = (MessageRecipient) o;
return recipient.equals(that.recipient);
}
void sendMessage(Context context, SignalServiceCallMessage callMessage)
throws IOException, UntrustedIdentityException, IOException
{
messageSender.sendCallMessage(new SignalServiceAddress(recipient.getAddress().toPhoneString()),
UnidentifiedAccessUtil.getAccessFor(context, recipient),
callMessage);
}
@Override
public void sendOfferMessage(Context context, long callId, String description)
throws IOException, UntrustedIdentityException, IOException
{
Log.i(TAG, "MessageRecipient::sendOfferMessage(): callId: " + callId);
OfferMessage offerMessage = new OfferMessage(callId, description);
sendMessage(context, SignalServiceCallMessage.forOffer(offerMessage));
}
@Override
public void sendAnswerMessage(Context context, long callId, String description)
throws IOException, UntrustedIdentityException, IOException
{
Log.i(TAG, "MessageRecipient::sendAnswerMessage(): callId: " + callId);
AnswerMessage answerMessage = new AnswerMessage(callId, description);
sendMessage(context, SignalServiceCallMessage.forAnswer(answerMessage));
}
@Override
public void sendIceUpdates(Context context, List<IceUpdateMessage> iceUpdateMessages)
throws IOException, UntrustedIdentityException, IOException
{
Log.i(TAG, "MessageRecipient::sendIceUpdates(): iceUpdates: " + iceUpdateMessages.size());
sendMessage(context, SignalServiceCallMessage.forIceUpdates(iceUpdateMessages));
}
@Override
public void sendHangupMessage(Context context, long callId)
throws IOException, UntrustedIdentityException, IOException
{
Log.i(TAG, "MessageRecipient::sendHangupMessage(): callId: " + callId);
HangupMessage hangupMessage = new HangupMessage(callId);
sendMessage(context, SignalServiceCallMessage.forHangup(hangupMessage));
}
@Override
public void sendBusyMessage(Context context, long callId)
throws IOException, UntrustedIdentityException, IOException
{
Log.i(TAG, "MessageRecipient::sendBusyMessage(): callId: " + callId);
BusyMessage busyMessage = new BusyMessage(callId);
sendMessage(context, SignalServiceCallMessage.forBusy(busyMessage));
}
}

843
src/org/thoughtcrime/securesms/service/WebRtcCallService.java
View File

File diff suppressed because it is too large Load Diff

11
src/org/thoughtcrime/securesms/webrtc/PeerConnectionFactoryOptions.java
View File

@ -1,11 +0,0 @@
package org.thoughtcrime.securesms.webrtc;
import org.webrtc.PeerConnectionFactory;
public class PeerConnectionFactoryOptions extends PeerConnectionFactory.Options {
public PeerConnectionFactoryOptions() {
this.networkIgnoreMask = 1 << 4;
}
}

424
src/org/thoughtcrime/securesms/webrtc/PeerConnectionWrapper.java
View File

@ -1,424 +0,0 @@
package org.thoughtcrime.securesms.webrtc;
import android.content.Context;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import org.thoughtcrime.securesms.logging.Log;
import org.thoughtcrime.securesms.util.concurrent.SettableFuture;
import org.webrtc.AudioSource;
import org.webrtc.AudioTrack;
import org.webrtc.Camera1Enumerator;
import org.webrtc.Camera2Enumerator;
import org.webrtc.CameraEnumerator;
import org.webrtc.CameraVideoCapturer;
import org.webrtc.DataChannel;
import org.webrtc.EglBase;
import org.webrtc.IceCandidate;
import org.webrtc.MediaConstraints;
import org.webrtc.MediaStream;
import org.webrtc.PeerConnection;
import org.webrtc.PeerConnectionFactory;
import org.webrtc.SdpObserver;
import org.webrtc.SessionDescription;
import org.webrtc.SurfaceTextureHelper;
import org.webrtc.VideoSink;
import org.webrtc.VideoSource;
import org.webrtc.VideoTrack;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.ExecutionException;
import static org.thoughtcrime.securesms.webrtc.CameraState.Direction.BACK;
import static org.thoughtcrime.securesms.webrtc.CameraState.Direction.FRONT;
import static org.thoughtcrime.securesms.webrtc.CameraState.Direction.NONE;
import static org.thoughtcrime.securesms.webrtc.CameraState.Direction.PENDING;
public class PeerConnectionWrapper {
private static final String TAG = PeerConnectionWrapper.class.getSimpleName();
private static final PeerConnection.IceServer STUN_SERVER = new PeerConnection.IceServer("stun:stun1.l.google.com:19302");
@NonNull private final PeerConnection peerConnection;
@NonNull private final AudioTrack audioTrack;
@NonNull private final AudioSource audioSource;
@NonNull private final Camera camera;
@Nullable private final VideoSource videoSource;
@Nullable private final VideoTrack videoTrack;
public PeerConnectionWrapper(@NonNull Context context,
@NonNull PeerConnectionFactory factory,
@NonNull PeerConnection.Observer observer,
@NonNull VideoSink localRenderer,
@NonNull List<PeerConnection.IceServer> turnServers,
@NonNull CameraEventListener cameraEventListener,
@NonNull EglBase eglBase,
boolean hideIp)
{
List<PeerConnection.IceServer> iceServers = new LinkedList<>();
iceServers.add(STUN_SERVER);
iceServers.addAll(turnServers);
MediaConstraints constraints = new MediaConstraints();
MediaConstraints audioConstraints = new MediaConstraints();
PeerConnection.RTCConfiguration configuration = new PeerConnection.RTCConfiguration(iceServers);
configuration.bundlePolicy = PeerConnection.BundlePolicy.MAXBUNDLE;
configuration.rtcpMuxPolicy = PeerConnection.RtcpMuxPolicy.REQUIRE;
if (hideIp) {
configuration.iceTransportsType = PeerConnection.IceTransportsType.RELAY;
}
constraints.optional.add(new MediaConstraints.KeyValuePair("DtlsSrtpKeyAgreement", "true"));
audioConstraints.optional.add(new MediaConstraints.KeyValuePair("DtlsSrtpKeyAgreement", "true"));
this.peerConnection = factory.createPeerConnection(configuration, constraints, observer);
this.peerConnection.setAudioPlayout(false);
this.peerConnection.setAudioRecording(false);
MediaStream mediaStream = factory.createLocalMediaStream("ARDAMS");
this.audioSource = factory.createAudioSource(audioConstraints);
this.audioTrack = factory.createAudioTrack("ARDAMSa0", audioSource);
this.audioTrack.setEnabled(false);
mediaStream.addTrack(audioTrack);
this.camera = new Camera(context, cameraEventListener);
if (camera.capturer != null) {
this.videoSource = factory.createVideoSource(false);
this.videoTrack = factory.createVideoTrack("ARDAMSv0", videoSource);
camera.capturer.initialize(SurfaceTextureHelper.create("WebRTC-SurfaceTextureHelper", eglBase.getEglBaseContext()), context, videoSource.getCapturerObserver());
this.videoTrack.addSink(localRenderer);
this.videoTrack.setEnabled(false);
mediaStream.addTrack(videoTrack);
} else {
this.videoSource = null;
this.videoTrack = null;
}
this.peerConnection.addStream(mediaStream);
}
public void setVideoEnabled(boolean enabled) {
if (this.videoTrack != null) {
this.videoTrack.setEnabled(enabled);
}
camera.setEnabled(enabled);
}
public void flipCamera() {
camera.flip();
}
public CameraState getCameraState() {
return new CameraState(camera.getActiveDirection(), camera.getCount());
}
public void setCommunicationMode() {
this.peerConnection.setAudioPlayout(true);
this.peerConnection.setAudioRecording(true);
}
public void setAudioEnabled(boolean enabled) {
this.audioTrack.setEnabled(enabled);
}
public DataChannel createDataChannel(String name) {
DataChannel.Init dataChannelConfiguration = new DataChannel.Init();
dataChannelConfiguration.ordered = true;
return this.peerConnection.createDataChannel(name, dataChannelConfiguration);
}
public SessionDescription createOffer(MediaConstraints mediaConstraints) throws PeerConnectionException {
final SettableFuture<SessionDescription> future = new SettableFuture<>();
peerConnection.createOffer(new SdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sdp) {
future.set(sdp);
}
@Override
public void onCreateFailure(String error) {
future.setException(new PeerConnectionException(error));
}
@Override
public void onSetSuccess() {
throw new AssertionError();
}
@Override
public void onSetFailure(String error) {
throw new AssertionError();
}
}, mediaConstraints);
try {
return correctSessionDescription(future.get());
} catch (InterruptedException e) {
throw new AssertionError(e);
} catch (ExecutionException e) {
throw new PeerConnectionException(e);
}
}
public SessionDescription createAnswer(MediaConstraints mediaConstraints) throws PeerConnectionException {
final SettableFuture<SessionDescription> future = new SettableFuture<>();
peerConnection.createAnswer(new SdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sdp) {
future.set(sdp);
}
@Override
public void onCreateFailure(String error) {
future.setException(new PeerConnectionException(error));
}
@Override
public void onSetSuccess() {
throw new AssertionError();
}
@Override
public void onSetFailure(String error) {
throw new AssertionError();
}
}, mediaConstraints);
try {
return correctSessionDescription(future.get());
} catch (InterruptedException e) {
throw new AssertionError(e);
} catch (ExecutionException e) {
throw new PeerConnectionException(e);
}
}
public void setRemoteDescription(SessionDescription sdp) throws PeerConnectionException {
final SettableFuture<Boolean> future = new SettableFuture<>();
peerConnection.setRemoteDescription(new SdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sdp) {}
@Override
public void onCreateFailure(String error) {}
@Override
public void onSetSuccess() {
future.set(true);
}
@Override
public void onSetFailure(String error) {
future.setException(new PeerConnectionException(error));
}
}, sdp);
try {
future.get();
} catch (InterruptedException e) {
throw new AssertionError(e);
} catch (ExecutionException e) {
throw new PeerConnectionException(e);
}
}
public void setLocalDescription(SessionDescription sdp) throws PeerConnectionException {
final SettableFuture<Boolean> future = new SettableFuture<>();
peerConnection.setLocalDescription(new SdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sdp) {
throw new AssertionError();
}
@Override
public void onCreateFailure(String error) {
throw new AssertionError();
}
@Override
public void onSetSuccess() {
future.set(true);
}
@Override
public void onSetFailure(String error) {
future.setException(new PeerConnectionException(error));
}
}, sdp);
try {
future.get();
} catch (InterruptedException e) {
throw new AssertionError(e);
} catch (ExecutionException e) {
throw new PeerConnectionException(e);
}
}
public void dispose() {
this.camera.dispose();
if (this.videoSource != null) {
this.videoSource.dispose();
}
this.audioSource.dispose();
this.peerConnection.close();
this.peerConnection.dispose();
}
public boolean addIceCandidate(IceCandidate candidate) {
return this.peerConnection.addIceCandidate(candidate);
}
private SessionDescription correctSessionDescription(SessionDescription sessionDescription) {
String updatedSdp = sessionDescription.description.replaceAll("(a=fmtp:111 ((?!cbr=).)*)\r?\n", "$1;cbr=1\r\n");
updatedSdp = updatedSdp.replaceAll(".+urn:ietf:params:rtp-hdrext:ssrc-audio-level.*\r?\n", "");
return new SessionDescription(sessionDescription.type, updatedSdp);
}
public static class PeerConnectionException extends Exception {
public PeerConnectionException(String error) {
super(error);
}
public PeerConnectionException(Throwable throwable) {
super(throwable);
}
}
private static class Camera implements CameraVideoCapturer.CameraSwitchHandler {
@Nullable
private final CameraVideoCapturer capturer;
private final CameraEventListener cameraEventListener;
private final int cameraCount;
private CameraState.Direction activeDirection;
private boolean enabled;
Camera(@NonNull Context context, @NonNull CameraEventListener cameraEventListener)
{
this.cameraEventListener = cameraEventListener;
CameraEnumerator enumerator = getCameraEnumerator(context);
cameraCount = enumerator.getDeviceNames().length;
CameraVideoCapturer capturerCandidate = createVideoCapturer(enumerator, FRONT);
if (capturerCandidate != null) {
activeDirection = FRONT;
} else {
capturerCandidate = createVideoCapturer(enumerator, BACK);
if (capturerCandidate != null) {
activeDirection = BACK;
} else {
activeDirection = NONE;
}
}
capturer = capturerCandidate;
}
void flip() {
if (capturer == null || cameraCount < 2) {
Log.w(TAG, "Tried to flip the camera, but we only have " + cameraCount + " of them.");
return;
}
activeDirection = PENDING;
capturer.switchCamera(this);
}
void setEnabled(boolean enabled) {
this.enabled = enabled;
if (capturer == null) {
return;
}
try {
if (enabled) {
capturer.startCapture(1280, 720, 30);
} else {
capturer.stopCapture();
}
} catch (InterruptedException e) {
Log.w(TAG, "Got interrupted while trying to stop video capture", e);
}
}
void dispose() {
if (capturer != null) {
capturer.dispose();
}
}
int getCount() {
return cameraCount;
}
@NonNull CameraState.Direction getActiveDirection() {
return enabled ? activeDirection : NONE;
}
@Nullable CameraVideoCapturer getCapturer() {
return capturer;
}
private @Nullable CameraVideoCapturer createVideoCapturer(@NonNull CameraEnumerator enumerator,
@NonNull CameraState.Direction direction)
{
String[] deviceNames = enumerator.getDeviceNames();
for (String deviceName : deviceNames) {
if ((direction == FRONT && enumerator.isFrontFacing(deviceName)) ||
(direction == BACK && enumerator.isBackFacing(deviceName)))
{
return enumerator.createCapturer(deviceName, null);
}
}
return null;
}
private @NonNull CameraEnumerator getCameraEnumerator(@NonNull Context context) {
boolean camera2EnumeratorIsSupported = false;
try {
camera2EnumeratorIsSupported = Camera2Enumerator.isSupported(context);
} catch (final Throwable throwable) {
Log.w(TAG, "Camera2Enumator.isSupport() threw.", throwable);
}
Log.i(TAG, "Camera2 enumerator supported: " + camera2EnumeratorIsSupported);
return camera2EnumeratorIsSupported ? new Camera2Enumerator(context)
: new Camera1Enumerator(true);
}
@Override
public void onCameraSwitchDone(boolean isFrontFacing) {
activeDirection = isFrontFacing ? FRONT : BACK;
cameraEventListener.onCameraSwitchCompleted(new CameraState(getActiveDirection(), getCount()));
}
@Override
public void onCameraSwitchError(String errorMessage) {
Log.e(TAG, "onCameraSwitchError: " + errorMessage);
cameraEventListener.onCameraSwitchCompleted(new CameraState(getActiveDirection(), getCount()));
}
}
public interface CameraEventListener {
void onCameraSwitchCompleted(@NonNull CameraState newCameraState);
}
}

2248
src/org/thoughtcrime/securesms/webrtc/WebRtcDataProtos.java
View File

File diff suppressed because it is too large Load Diff