Case Analysis of System-wide Stutter Caused by Android Accessibility Service

Word count: 1.9kReading time: 11 min

 2019/01/21

Phenomenon

Some users reported that while scrolling on the phone, the list would jitter. This happens when scrolling on the desktop or settings (as long as it is scrollable), but this is not reproducible for everyone; it appears for some users, but not for others.

Onlookers can scroll directly to the bottom to see The Culprit and Self-Check. Those interested in analyzing the problem can read the analysis process.

Systrace Analysis

There is a reproducible device for local testing. After analyzing it, it was found that sliding the desktop or settings with a finger will inevitably cause stuttering. From the Trace, it looks like this:

The red arrow indicates where the frame drop occurred. From the Buffer count above, it can be seen that the reason SF did not draw is that Launcher did not submit a Buffer.

The corresponding Launcher Trace is as follows. It can be seen that the reason Launcher did not draw is that no Input event was passed up. So the Launcher’s screen was not updated, causing frame drops.

The fact that no event came up is problematic in itself. Our finger slides across the screen continuously, so the event reporting should be continuous. We suspect there is a problem with the screen reporting points, but before checking the hardware, let’s first see if the InputReader and InputDispatcher threads are working properly.

From the figure, it can be seen that the InputReader thread is working properly, but the InputDispatcher thread has problems. You can see the correspondence between these two threads under normal circumstances:

Returning to the problematic figure, if you look closely, you will find that the cycle of the InputDispatcher thread is the same as Vsync. That is to say, the wake-up logic of InputDispatcher has changed from being woken up by InputReader to being woken up by Vsync.

If you look more closely, clicking on the cpu state of the InputDispatcher thread shows that the InputDispatcher thread waking up to execute tasks is not woken up by the InputReader thread, but by the UI Thread of System_Server.

So next, we need to look at why InputReader did not wake up InputDispatcher from the code perspective.

Code Analysis

The logic for InputReader to wake up the InputDispatcher thread is as follows (taking the Move gesture in this example accurately).

frameworks/native/services/inputflinger/InputDispatcher.cpp

void InputDispatcher::notifyMotion(const NotifyMotionArgs* args) {

    if (!validateMotionEvent(args->action, args->actionButton,
                args->pointerCount, args->pointerProperties)) {
        return;
    }

    uint32_t policyFlags = args->policyFlags;
    policyFlags |= POLICY_FLAG_TRUSTED;

    android::base::Timer t;
    mPolicy->interceptMotionBeforeQueueing(args->eventTime, /*byref*/ policyFlags);
    if (t.duration() > SLOW_INTERCEPTION_THRESHOLD) {
        ALOGW("Excessive delay in interceptMotionBeforeQueueing; took %s ms",
                std::to_string(t.duration().count()).c_str());
    }

    bool needWake; // Whether to wake up
    { // acquire lock
        mLock.lock();

        if (shouldSendMotionToInputFilterLocked(args)) {
            mLock.unlock();

            MotionEvent event;
            event.initialize(args->deviceId, args->source, args->action, args->actionButton,
                    args->flags, args->edgeFlags, args->metaState, args->buttonState,
                    0, 0, args->xPrecision, args->yPrecision,
                    args->downTime, args->eventTime,
                    args->pointerCount, args->pointerProperties, args->pointerCoords);

            policyFlags |= POLICY_FLAG_FILTERED;
            // SystemServer upper layer needs to filter events
            if (!mPolicy->filterInputEvent(&event, policyFlags)) {
                return; // event was consumed by the filter
            }

            mLock.lock();
        }

        // Just enqueue a new motion event.
        MotionEntry* newEntry = new MotionEntry(args->eventTime,
                args->deviceId, args->source, policyFlags,
                args->action, args->actionButton, args->flags,
                args->metaState, args->buttonState,
                args->edgeFlags, args->xPrecision, args->yPrecision, args->downTime,
                args->displayId,
                args->pointerCount, args->pointerProperties, args->pointerCoords, 0, 0);

        needWake = enqueueInboundEventLocked(newEntry); 
        mLock.unlock();
    } // release lock

    if (needWake) {
        mLooper->wake();
    }
}

Note here that mPolicy->filterInputEvent returns directly, which means if it returns false here, it returns directly and does not continue to execute the following steps.

Continue to look at mPolicy->filterInputEvent

frameworks/base/services/core/jni/com_android_server_input_InputManagerService.cpp

bool NativeInputManager::filterInputEvent(const InputEvent* inputEvent, uint32_t policyFlags) {
    ATRACE_CALL();
    jobject inputEventObj;

    JNIEnv* env = jniEnv();
    switch (inputEvent->getType()) {
    case AINPUT_EVENT_TYPE_KEY:
        inputEventObj = android_view_KeyEvent_fromNative(env,
                static_cast<const KeyEvent*>(inputEvent));
        break;
    case AINPUT_EVENT_TYPE_MOTION:
        inputEventObj = android_view_MotionEvent_obtainAsCopy(env,
                static_cast<const MotionEvent*>(inputEvent));
        break;
    default:
        return true; // dispatch the event normally
    }

     
    
    // The callee is responsible for recycling the event.
    jboolean pass = env->CallBooleanMethod(mServiceObj, gServiceClassInfo.filterInputEvent,
            inputEventObj, policyFlags);
    if (checkAndClearExceptionFromCallback(env, "filterInputEvent")) {
        pass = true;
    }
    env->DeleteLocalRef(inputEventObj);
    return pass;
}

Here it calls back from jni to the java layer, which is the filterInputEvent method of InputManagerService.

com/android/server/input/InputManagerService.java

// Native callback.
final boolean filterInputEvent(InputEvent event, int policyFlags) {
    synchronized (mInputFilterLock) {
        if (mInputFilter != null) {
            try {
                mInputFilter.filterInputEvent(event, policyFlags);
            } catch (RemoteException e) {
                /* ignore */
            }
            return false;
        }
    }
    event.recycle();
    return true;
}

Following the code flow, it is found that this mInputFilter is an instance of AccessibilityInputFilter. When the switch is turned on in Accessibility, the updateInputFilter method of AccessibilityManagerService is called to set the InputFilter.

android/view/InputFilter.java

final public void filterInputEvent(InputEvent event, int policyFlags) {
    mH.obtainMessage(MSG_INPUT_EVENT, policyFlags, 0, event).sendToTarget();
}

case MSG_INPUT_EVENT: {
    final InputEvent event = (InputEvent)msg.obj;
    try {
        if (mInboundInputEventConsistencyVerifier != null) {
            mInboundInputEventConsistencyVerifier.onInputEvent(event, 0);
        }
        onInputEvent(event, msg.arg1);
    } finally {
        event.recycle();
    }
    break;
}

Continue to look at onInputEvent(event, msg.arg1);
com/android/server/accessibility/AccessibilityInputFilter.java

@Override
public void onInputEvent(InputEvent event, int policyFlags) {
    if (mEventHandler == null) {
        if (DEBUG) Slog.d(TAG, "mEventHandler == null for event " + event);
        super.onInputEvent(event, policyFlags);
        return;
    }

    EventStreamState state = getEventStreamState(event);
    if (state == null) {
        super.onInputEvent(event, policyFlags);
        return;
    }

    int eventSource = event.getSource();
    if ((policyFlags & WindowManagerPolicy.FLAG_PASS_TO_USER) == 0) {
        state.reset();
        mEventHandler.clearEvents(eventSource);
        super.onInputEvent(event, policyFlags);
        return;
    }

    if (state.updateDeviceId(event.getDeviceId())) {
        mEventHandler.clearEvents(eventSource);
    }

    if (!state.deviceIdValid()) {
        super.onInputEvent(event, policyFlags);
        return;
    }

    if (event instanceof MotionEvent) {
        if ((mEnabledFeatures & FEATURES_AFFECTING_MOTION_EVENTS) != 0) {
            MotionEvent motionEvent = (MotionEvent) event;
            processMotionEvent(state, motionEvent, policyFlags);
            return;
        } else {
            super.onInputEvent(event, policyFlags);
        }
    } else if (event instanceof KeyEvent) {
        KeyEvent keyEvent = (KeyEvent) event;
        processKeyEvent(state, keyEvent, policyFlags);
    }
}

Continue to look at processMotionEvent

private void processMotionEvent(EventStreamState state, MotionEvent event, int policyFlags) {
    if (!state.shouldProcessScroll() && event.getActionMasked() == MotionEvent.ACTION_SCROLL) {
        super.onInputEvent(event, policyFlags);
        return;
    }

    if (!state.shouldProcessMotionEvent(event)) {
        return;
    }

    batchMotionEvent(event, policyFlags);
}

Continue to look at batchMotionEvent

private void batchMotionEvent(MotionEvent event, int policyFlags) {
    if (DEBUG) {
        Slog.i(TAG, "Batching event: " + event + ", policyFlags: " + policyFlags);
    }
    if (mEventQueue == null) {
        mEventQueue = MotionEventHolder.obtain(event, policyFlags);
        scheduleProcessBatchedEvents();
        return;
    }
    if (mEventQueue.event.addBatch(event)) {
        return;
    }
    MotionEventHolder holder = MotionEventHolder.obtain(event, policyFlags);
    holder.next = mEventQueue;
    mEventQueue.previous = holder;
    mEventQueue = holder;
}

Continue to look at scheduleProcessBatchedEvents

private void scheduleProcessBatchedEvents() {
    mChoreographer.postCallback(Choreographer.CALLBACK_INPUT,
            mProcessBatchedEventsRunnable, null);
}

mProcessBatchedEventsRunnable will be executed in the next Vsync cycle, which is Choreographer.CALLBACK_INPUT. Students familiar with Choregrapher should know what is happening here.

private final Runnable mProcessBatchedEventsRunnable = new Runnable() {
    @Override
    public void run() {
        final long frameTimeNanos = mChoreographer.getFrameTimeNanos();
        }
        processBatchedEvents(frameTimeNanos);
        if (mEventQueue != null) {
            scheduleProcessBatchedEvents();
        }
    }
};

So the code is relatively clear here. It is because of the existence of AccessibilityInputFilter that the InputDispatcher thread was not woken up, but the event processing was put into the next Vsync for processing.
Conclusion

The problem lies in this Runnable. Normally, if AccessibilityInputFilter is not turned on, the upper layer will not intercept Input events at all. Once there is AccessibilityInputFilter, the logic above will be followed. At this time, InputDispatcher will not follow the rhythm of InputReader, but follow the rhythm of Vsync. This can also be seen from the Trace;

So where does this AccessibilityInputFilter come from? The answer is the Accessibility service, also known as the barrier-free service.

The Culprit

Through the above analysis, we know that the cause of the problem is the Accessibility service. The essence of the Accessibility service is to serve those users who are inconvenient to operate, such as blind people. However, in order to implement specific functions, some Apps have also added their own Accessibility services, such as the “one-click installation” function of major mobile phone markets. It is convenient for users, but if not used well, it will also have negative effects, such as this case, causing the user’s mobile phone to stutter. Users who don’t know may want to return the phone.

So who is the culprit? Currently two have been found, one is iFlytek Input Method and the other is Yingyongbao. Open Settings - System - Accessibility, you can see that various software inside are involved, but this is turned off by default. Many applications will guide users to turn it on, and many users turn it on confusedly.

The Accessibility service page is as follows:

As for how awesome the Accessibility service is, everyone can look at the pop-up box below. This thing can detect your credit card number and password. As for SMS content and WeChat chat content, those are small cases.

As for what caused the whole machine to stutter in this example, it is the following listener “Perform Gesture”. Once an application listens to this, the InputDispatcher thread will follow the cycle of Vsync, resulting in untimely report processing, causing the sliding object to think that no event entered this frame, so there is no content change, and the page will not be updated, resulting in stuttering.

Self-Check

If you are using an Android phone, it is strongly recommended that you turn off all Accessibility services (if you don’t need them). Functions like automatic application installation are not worth the huge risk you pay for them. This is a native Android problem, and we have found this problem on Pixel and other third-party phones.

Close path: Settings - System - Accessibility, go in and close all the ones you have opened.
Strongly recommend Yingyongbao, iFlytek Input Method, do not listen to gesture events.

Zhihu Address of this article

Since blog message communication is inconvenient, for likes or communication, please move to the Zhihu interface of this article
Zhihu - Analysis of Global Stutter Caused by Accessibility Services of Yingyongbao and iFlytek Input Method on Android Platform

About Me && Blog

Below are personal introduction and related links. I hope to communicate more with everyone in the industry. If three people walk together, there must be one who can be my teacher!